You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/03/07 07:40:19 UTC

[GitHub] [pinot] KKcorps opened a new pull request #8304: Add commonly used math, string and date scalar functions in Pinot

KKcorps opened a new pull request #8304:
URL: https://github.com/apache/pinot/pull/8304


   There are some functions which are missing from Pinot currently which are part of standard SQL. We can still do some of these operations (except for trignometric ones) by writing composite SQL expressions using existing functions. This however leads to complex queries and it is better we support these functions directly in Pinot.
   This also helps in providing better support for third party tools such as Tableau.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827358617



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       I don't think that's the gold standard though. If some BI tool generates `log` it expects a natural logarithm, and it's unlikely to generate `ln`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828282452



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {

Review comment:
       fixed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827351091



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       I referenced the presto functions: https://prestodb.io/docs/current/functions/math.html




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r831197249



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -164,7 +164,7 @@ protected void applyMathOperator(double[] values, int length) {
 
   public static class Log2TransformFunction extends SingleParamMathTransformFunction {
     public static final String FUNCTION_NAME = "log2";
-    public static final double LOG_BASE = Math.log(2);
+    public static final double LOG_BASE = 0.6931471805599453d;

Review comment:
       done.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1071315988


   @Jackie-Jiang @richardstartin I have added the unit-tests as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r835615364



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       I like the idea of using a boolean flag, so that it is guaranteed to have the desired behavior without potentially paying the cost of unboxing the values




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r835754532



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       Yeah, boolean flags are better. using them.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e81d81c) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `0.95%`.
   > The diff coverage is `77.24%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   69.81%   -0.96%     
   + Complexity     4278     4277       -1     
   ============================================
     Files          1655     1660       +5     
     Lines         86665    86946     +281     
     Branches      13067    13117      +50     
   ============================================
   - Hits          61327    60698     -629     
   - Misses        21095    22014     +919     
   + Partials       4243     4234       -9     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.67% <14.13%> (+<0.01%)` | :arrow_up: |
   | integration2 | `?` | |
   | unittests1 | `67.03% <77.24%> (+0.05%)` | :arrow_up: |
   | unittests2 | `14.08% <0.00%> (-0.08%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `69.56% <37.50%> (+16.23%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `71.05% <64.28%> (-2.80%)` | :arrow_down: |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `72.50% <72.50%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [108 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...e81d81c](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r820500243



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {

Review comment:
       These can all be autovectorized on recent JVMs on x86, but need to be implemented as `TransformFunction`s. I would suggest generating the code with a one off script.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825762043



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }

Review comment:
        The reason I had to do it this way because we want to integrate it with Tableau and Tableau expects these as functions e.g. PI() instead of PI. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826839612



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       Having said that, there are log10 and log2 in various databases, so we should add these. It seems not all databases allow arbitrary bases either.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] mneedham commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
mneedham commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828963670



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @Jackie-Jiang if you have the same function as a scalar function and transform function, how does Pinot decide which one to use? Or would they need to have different names?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r829106046



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in TransformFunctionFactory ensures this order.
   
   ```java
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r829106046



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in TransformFunctionFactory ensures this order.
   
   ```
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830273744



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];

Review comment:
       It is, but we’ve been trying to migrate away from defaulting to 10k for a while now, maybe someone should go through all of them and fix it. This is where we see most “humongous” allocations coming from in production.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830180752



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];

Review comment:
       this applies to all functions




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826588397



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       I think it is still needed. e.g. https://github.com/apache/pinot/blob/master/pinot-tools/src/main/resources/examples/batch/githubEvents/githubEvents_offline_complexTypeHandling_table_config.json
   In this what happens to `created_at_timestamp`, when `created_at` is null/empty ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828545880



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -57,12 +57,20 @@
   public LiteralTransformFunction(String literal) {
     _literal = literal;
     _dataType = inferLiteralDataType(literal);
+    double mathLiteral;
     if (_dataType.isNumeric()) {
       BigDecimal bigDecimal = new BigDecimal(_literal);
       _intLiteral = bigDecimal.intValue();
       _longLiteral = bigDecimal.longValue();
       _floatLiteral = bigDecimal.floatValue();
       _doubleLiteral = bigDecimal.doubleValue();
+    } else if ((mathLiteral = getMathConstant(literal)) != 0) {

Review comment:
       Blindly treating `E` and `PI` as number can cause unexpected behavior. They should be parsed within the math function context. Also I'd suggest addressing it in a separate PR.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826837137



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       This function should actually take a base as an optional parameter to be sql compliant  (I understand that this mechanism doesn’t support optional parameters, but a default value could possibly be used.)
   
   If the base parameter is default, a natural logarithm should be computed. If it is 10 or 2, the appropriate math functions should be called. If it is something else it should be calculated as follows: 
   
   logb(x) = log10(x) / log10(b)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ccbba40) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `6.57%`.
   > The diff coverage is `77.16%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   64.19%   -6.58%     
   - Complexity     4278     4283       +5     
   ============================================
     Files          1655     1616      -39     
     Lines         86665    85165    -1500     
     Branches      13067    12931     -136     
   ============================================
   - Hits          61327    54669    -6658     
   - Misses        21095    26546    +5451     
   + Partials       4243     3950     -293     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `67.07% <77.16%> (+0.09%)` | :arrow_up: |
   | unittests2 | `14.15% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `65.21% <37.50%> (+11.88%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `69.73% <64.28%> (-4.11%)` | :arrow_down: |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [394 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...ccbba40](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r820497440



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       `@ScalarFunction` is convenient but inefficient. These operations can all be autovectorized, but only if they are implemented as `TransformFunction`s. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828414465



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {

Review comment:
       cool. renamed this function and added another one with limit.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826402604



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       This is also required for post-aggregation (etc. `SELECT log10(sum(col1)) FROM myTable`) which is part of SQL, so we need to ensure the scalar function and transform function matches.
   
   So I'd suggest doing it following these steps:
   1. Add the scalar function version so that the query works (performance won't be very good without proper vectorization)
   2. Add the transform function version to optimize the performance 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825772219



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       I’m not sure, every time I’ve seen it used, the scalar function wrapper TransformFunction appears to be highly inefficient, so this is more of a question of usability than an “icing on the cake” kind of thing. We want users to both have access to these functions and be able to use them in real applications.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825762043



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }

Review comment:
        The reason I had to do it this way because we want to integrate it with Tableau and Tableau expects these as functions e.g. PI() instead of PI. I am making changes in Tableau dialect file as well for our connector post-which we can change this to Literal.

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       +1




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825767639



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {

Review comment:
       Cotangent (cos * sin)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828766021



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -57,12 +57,20 @@
   public LiteralTransformFunction(String literal) {
     _literal = literal;
     _dataType = inferLiteralDataType(literal);
+    double mathLiteral;
     if (_dataType.isNumeric()) {
       BigDecimal bigDecimal = new BigDecimal(_literal);
       _intLiteral = bigDecimal.intValue();
       _longLiteral = bigDecimal.longValue();
       _floatLiteral = bigDecimal.floatValue();
       _doubleLiteral = bigDecimal.doubleValue();
+    } else if ((mathLiteral = getMathConstant(literal)) != 0) {

Review comment:
       Ok. Yeah, I get for e.g. in a function like `startsWith(colA, 'E')` we should not be replacing `E` with Euler's consntat.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830179375



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+    }
+
+    int length = projectionBlock.getNumDocs();
+    double[] leftValues = _leftTransformFunction.transformToDoubleValuesSV(projectionBlock);
+    double[] rightValues = _rightTransformFunction.transformToDoubleValuesSV(projectionBlock);
+    for (int i = 0; i < length; i++) {
+      _result[i] = Math.pow(leftValues[i], rightValues[i]);

Review comment:
       Here the power will usually be a constant. Can you check if the RHS is `LiteralTransformFunction` in `init` and set a `_power` variable to `Double.parseDouble(((LiteralTransformFunction) arguments.get(1)).getLiteral())` and set a flag that the power is trivial, then if the power is trivial just do `_result[i] = Math.pow(leftValues[i], _power);` here instead. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830180329



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
##########
@@ -0,0 +1,100 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+//TODO: The function should ideally be named 'round'
+// but it is not possible because of existing DateTimeFunction with same name.
+public class RoundDecimalTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "roundDecimal";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    int numArguments = arguments.size();
+    // Check that there are more than 2 arguments or no arguments
+    if (numArguments < 1 || numArguments > 2) {
+      throw new IllegalArgumentException(
+          "roundDecimal transform function supports either 1 or 2 arguments. Num arguments provided: " + numArguments);
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    if (numArguments > 1) {
+      _rightTransformFunction = arguments.get(1);
+      Preconditions.checkArgument(_rightTransformFunction.getResultMetadata().isSingleValue()
+              && isArgumentDataTypeValid(_rightTransformFunction),
+          "Argument must be single-valued with type INT for transform function: %s", getName());
+    } else {
+      _rightTransformFunction = null;
+    }
+
+    Preconditions.checkArgument(_leftTransformFunction.getResultMetadata().isSingleValue(),
+        "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  private boolean isArgumentDataTypeValid(TransformFunction transformFunction) {
+    return transformFunction.getResultMetadata().getDataType().getStoredType() == FieldSpec.DataType.INT
+        || transformFunction.getResultMetadata().getDataType().getStoredType() == FieldSpec.DataType.LONG;

Review comment:
       Suggesting adding a flag to `DataType` - `isIntegral` or something like that. It will be useful.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r832103839



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TrigonometricTransformFunctions.java
##########
@@ -0,0 +1,281 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class TrigonometricTransformFunctions {
+  public static class Atan2TransformFunction extends BaseTransformFunction {
+    public static final String FUNCTION_NAME = "atan2";
+    private double[] _result;
+    private TransformFunction _leftTransformFunction;
+    private TransformFunction _rightTransformFunction;
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+      // Check that there are more than 1 arguments
+      if (arguments.size() != 2) {
+        throw new IllegalArgumentException("Exactly 2 arguments are required for Atan2 transform function");
+      }
+
+      _leftTransformFunction = arguments.get(0);
+      _rightTransformFunction = arguments.get(1);
+      Preconditions.checkArgument(
+          _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+              .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+    }
+
+    @Override
+    public TransformResultMetadata getResultMetadata() {
+      return DOUBLE_SV_NO_DICTIONARY_METADATA;
+    }
+
+    @Override
+    public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+      int length = projectionBlock.getNumDocs();
+
+      if (_result == null || _result.length < length) {
+        _result = new double[length];
+      }
+
+      double[] leftValues = _leftTransformFunction.transformToDoubleValuesSV(projectionBlock);
+      double[] rightValues = _rightTransformFunction.transformToDoubleValuesSV(projectionBlock);
+      for (int i = 0; i < length; i++) {
+        _result[i] = Math.atan2(leftValues[i], rightValues[i]);
+      }
+
+      return _result;
+    }
+  }
+
+  public static class DegreesTransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "degrees";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.toDegrees(values[i]);
+      }
+      ;
+    }
+  }
+
+  public static class AcosTransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "acos";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.acos(values[i]);
+      }
+      ;

Review comment:
       Can we just get rid of these semicolons please?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r831053899



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];

Review comment:
       For now, I have made this change in the classes which are under scope of this PR. 
   I will raise a seperate PR for the rest of the functions.
   I want to know the reason though, why was it done this way. Maybe we are missing something?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+    }
+
+    int length = projectionBlock.getNumDocs();
+    double[] leftValues = _leftTransformFunction.transformToDoubleValuesSV(projectionBlock);
+    double[] rightValues = _rightTransformFunction.transformToDoubleValuesSV(projectionBlock);
+    for (int i = 0; i < length; i++) {
+      _result[i] = Math.pow(leftValues[i], rightValues[i]);

Review comment:
       done

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
##########
@@ -0,0 +1,100 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+//TODO: The function should ideally be named 'round'
+// but it is not possible because of existing DateTimeFunction with same name.
+public class RoundDecimalTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "roundDecimal";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    int numArguments = arguments.size();
+    // Check that there are more than 2 arguments or no arguments
+    if (numArguments < 1 || numArguments > 2) {
+      throw new IllegalArgumentException(
+          "roundDecimal transform function supports either 1 or 2 arguments. Num arguments provided: " + numArguments);
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    if (numArguments > 1) {
+      _rightTransformFunction = arguments.get(1);
+      Preconditions.checkArgument(_rightTransformFunction.getResultMetadata().isSingleValue()
+              && isArgumentDataTypeValid(_rightTransformFunction),
+          "Argument must be single-valued with type INT for transform function: %s", getName());
+    } else {
+      _rightTransformFunction = null;
+    }
+
+    Preconditions.checkArgument(_leftTransformFunction.getResultMetadata().isSingleValue(),
+        "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  private boolean isArgumentDataTypeValid(TransformFunction transformFunction) {
+    return transformFunction.getResultMetadata().getDataType().getStoredType() == FieldSpec.DataType.INT
+        || transformFunction.getResultMetadata().getDataType().getStoredType() == FieldSpec.DataType.LONG;

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834688916



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
##########
@@ -43,7 +43,14 @@
   EXP("exp"),
   FLOOR("floor"),
   LN("ln"),
+  LOG("log"),
   SQRT("sqrt"),
+  LOG2("log2"),

Review comment:
       I’ve changed the way the registry works, so now log and ln can be aliases for the same type.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834760315



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;

Review comment:
       This is not a set standard across classes. I can use it for this PR though but we should have a PR in future implementing this for rest of the classes.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834772884



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double roundDecimal(double a) {
+    return BigDecimal.valueOf(a).setScale(0, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double truncate(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.DOWN).doubleValue();

Review comment:
       Yeah you are right, I mixes `DOWN` and `FLOOR` mode




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825760858



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {

Review comment:
       I have kept it consistent with existing pinot queries. In Presto, every string function operates on the expectation that String index starts with 1. 
   Pinot's function on the other hand have String index 0.
   If we want to change it, we should change all of them or none of them.
   
   Also, changing them will break user queries who are already using existing functions.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825800911



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       I can implement all of them as transform functions while keeping them as Scalar functions as well. The current design limitation is both of these can't co-exist since mapping keys are created based only on names of the functions.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826843186



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       Postgres accepts a base https://database.guide/how-log-works-in-postgresql/ but sql server accepts the base in a different parameter position https://www.w3schools.com/sql/func_sqlserver_log.asp
   
   Maybe we can just ignore bases and add log (natural), log2 and log10.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834774315



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       Using boxed value is fine if we unbox the value before getting into the loop (not sure if JVM will do the optimization, cc @richardstartin)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ccbba40) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `56.60%`.
   > The diff coverage is `0.00%`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8304       +/-   ##
   =============================================
   - Coverage     70.76%   14.15%   -56.61%     
   + Complexity     4278       84     -4194     
   =============================================
     Files          1655     1616       -39     
     Lines         86665    85165     -1500     
     Branches      13067    12931      -136     
   =============================================
   - Hits          61327    12055    -49272     
   - Misses        21095    72201    +51106     
   + Partials       4243      909     -3334     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.15% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (-53.34%)` | :arrow_down: |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `0.00% <0.00%> (-95.96%)` | :arrow_down: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `0.00% <0.00%> (-73.85%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (-95.66%)` | :arrow_down: |
   | [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `0.00% <0.00%> (-85.94%)` | :arrow_down: |
   | ... and [1329 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...ccbba40](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps merged pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps merged pull request #8304:
URL: https://github.com/apache/pinot/pull/8304


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830181728



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -161,4 +161,55 @@ protected void applyMathOperator(double[] values, int length) {
       }
     }
   }
+
+  public static class Log2TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log2";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log(values[i]) / Math.log(2);

Review comment:
       suggest factoring `Math.log(2)` into a `private static final` constant. Not sure if this will fold otherwise.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1071315988


   @Jackie-Jiang @richardstartin I have added the unit-tests as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a7be2c2) into [master](https://codecov.io/gh/apache/pinot/commit/f67b5aacc5b494a2f5d78e87d67a84ea3aadc99a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f67b5aa) will **decrease** coverage by `34.37%`.
   > The diff coverage is `30.35%`.
   
   > :exclamation: Current head a7be2c2 differs from pull request most recent head 16939f0. Consider uploading reports for the commit 16939f0 to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8304/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8304       +/-   ##
   =============================================
   - Coverage     70.76%   36.39%   -34.38%     
   + Complexity     4244       81     -4163     
   =============================================
     Files          1631     1637        +6     
     Lines         85490    85857      +367     
     Branches      12878    12921       +43     
   =============================================
   - Hits          60499    31247    -29252     
   - Misses        20819    52103    +31284     
   + Partials       4172     2507     -1665     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.89% <24.60%> (+0.02%)` | :arrow_up: |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.17% <11.40%> (-0.04%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `40.00% <0.00%> (-60.00%)` | :arrow_down: |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `18.18% <0.00%> (-35.16%)` | :arrow_down: |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `3.33% <0.00%> (-92.59%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | `48.38% <0.00%> (-1.01%)` | :arrow_down: |
   | [...inot/core/operator/filter/FilterOperatorUtils.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRmlsdGVyT3BlcmF0b3JVdGlscy5qYXZh) | `68.75% <0.00%> (-16.25%)` | :arrow_down: |
   | [.../pinot/core/operator/filter/NotFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvTm90RmlsdGVyT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (-55.56%)` | :arrow_down: |
   | [.../main/java/org/apache/pinot/core/util/OsCheck.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL09zQ2hlY2suamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...exsegment/mutable/DefaultMutableIndexProvider.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9pbmRleHNlZ21lbnQvbXV0YWJsZS9EZWZhdWx0TXV0YWJsZUluZGV4UHJvdmlkZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | ... and [1029 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f67b5aa...16939f0](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830185003



##########
File path: pinot-core/src/test/java/org/apache/pinot/core/operator/transform/function/TrigonometricFunctionsTest.java
##########
@@ -0,0 +1,163 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import java.util.function.DoubleUnaryOperator;
+import org.apache.pinot.common.request.context.ExpressionContext;
+import org.apache.pinot.common.request.context.RequestContextUtils;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.AcosTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.AsinTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.Atan2TransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.AtanTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.CosTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.CoshTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.DegreesTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.RadiansTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.SinTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.SinhTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.TanTransformFunction;
+import org.apache.pinot.core.operator.transform.function.TrigonometricTransformFunctions.TanhTransformFunction;
+import org.testng.Assert;
+import org.testng.annotations.Test;
+
+
+public class TrigonometricFunctionsTest extends BaseTransformFunctionTest {

Review comment:
       🥇 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830184435



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunction.java
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+public class TruncateDecimalTransformFunction extends BaseTransformFunction {

Review comment:
       Again, I think the second argument will typically be a constant here, can you check if it's a literal and treat it accordingly?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830247631



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -161,4 +161,55 @@ protected void applyMathOperator(double[] values, int length) {
       }
     }
   }
+
+  public static class Log2TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log2";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log(values[i]) / Math.log(2);
+      }
+      ;
+    }
+  }
+
+  public static class Log10TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log10";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log10(values[i]);
+      }
+      ;

Review comment:
       wow. I guess intelliJ shortcuts with codestyle is definitely messing these up. I will refrain using those from now on.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834773380



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;

Review comment:
       Agree. We can have a separate PR to clean up the existing ones




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e81d81c) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `3.72%`.
   > The diff coverage is `77.24%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   67.03%   -3.73%     
   + Complexity     4278     4193      -85     
   ============================================
     Files          1655     1258     -397     
     Lines         86665    63485   -23180     
     Branches      13067     9918    -3149     
   ============================================
   - Hits          61327    42559   -18768     
   + Misses        21095    17880    -3215     
   + Partials       4243     3046    -1197     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `67.03% <77.24%> (+0.05%)` | :arrow_up: |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `65.21% <37.50%> (+11.88%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `69.73% <64.28%> (-4.11%)` | :arrow_down: |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `72.50% <72.50%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [639 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...e81d81c](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3c4cde7) into [master](https://codecov.io/gh/apache/pinot/commit/f67b5aacc5b494a2f5d78e87d67a84ea3aadc99a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f67b5aa) will **decrease** coverage by `6.60%`.
   > The diff coverage is `63.57%`.
   
   > :exclamation: Current head 3c4cde7 differs from pull request most recent head 9bdacb2. Consider uploading reports for the commit 9bdacb2 to get more accurate results
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   64.15%   -6.61%     
   - Complexity     4244     4252       +8     
   ============================================
     Files          1631     1597      -34     
     Lines         85490    84168    -1322     
     Branches      12878    12742     -136     
   ============================================
   - Hits          60499    54000    -6499     
   - Misses        20819    26280    +5461     
   + Partials       4172     3888     -284     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `67.00% <66.45%> (+0.03%)` | :arrow_up: |
   | unittests2 | `14.18% <9.59%> (-0.03%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...roker/requesthandler/GrpcBrokerRequestHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcmVxdWVzdGhhbmRsZXIvR3JwY0Jyb2tlclJlcXVlc3RIYW5kbGVyLmphdmE=) | `0.00% <0.00%> (-78.58%)` | :arrow_down: |
   | [...va/org/apache/pinot/common/config/NettyConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL05ldHR5Q29uZmlnLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...pache/pinot/common/config/provider/TableCache.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3Byb3ZpZGVyL1RhYmxlQ2FjaGUuamF2YQ==) | `0.00% <0.00%> (-83.86%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...g/apache/pinot/common/metrics/ControllerGauge.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyR2F1Z2UuamF2YQ==) | `0.00% <0.00%> (-97.30%)` | :arrow_down: |
   | [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | `13.30% <0.00%> (-36.09%)` | :arrow_down: |
   | [.../pinot/core/operator/filter/NotFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvTm90RmlsdGVyT3BlcmF0b3IuamF2YQ==) | `66.66% <ø> (+11.11%)` | :arrow_up: |
   | [...ry/optimizer/statement/JsonStatementOptimizer.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9vcHRpbWl6ZXIvc3RhdGVtZW50L0pzb25TdGF0ZW1lbnRPcHRpbWl6ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | ... and [459 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f67b5aa...9bdacb2](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827357081



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {

Review comment:
       My bad, cot is actually cos x **/** sin x or 1/tan x and I didn't check the implementation. https://en.wikipedia.org/wiki/Trigonometric_functions#Right-angled_triangle_definitions




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828434358



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -57,12 +57,20 @@
   public LiteralTransformFunction(String literal) {
     _literal = literal;
     _dataType = inferLiteralDataType(literal);
+    double mathLiteral;
     if (_dataType.isNumeric()) {
       BigDecimal bigDecimal = new BigDecimal(_literal);
       _intLiteral = bigDecimal.intValue();
       _longLiteral = bigDecimal.longValue();
       _floatLiteral = bigDecimal.floatValue();
       _doubleLiteral = bigDecimal.doubleValue();
+    } else if ((mathLiteral = getMathConstant(literal)) != 0) {

Review comment:
       @richardstartin is this approach fine or should I seperate this out entirely in a new class called `MathLiteralTransformFunctions` ?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e81d81c) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `6.65%`.
   > The diff coverage is `77.24%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   64.11%   -6.66%     
   + Complexity     4278     4277       -1     
   ============================================
     Files          1655     1615      -40     
     Lines         86665    85065    -1600     
     Branches      13067    12914     -153     
   ============================================
   - Hits          61327    54537    -6790     
   - Misses        21095    26579    +5484     
   + Partials       4243     3949     -294     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `67.03% <77.24%> (+0.05%)` | :arrow_up: |
   | unittests2 | `14.08% <0.00%> (-0.08%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `65.21% <37.50%> (+11.88%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `69.73% <64.28%> (-4.11%)` | :arrow_down: |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `72.50% <72.50%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [388 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...e81d81c](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834759693



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double roundDecimal(double a) {
+    return BigDecimal.valueOf(a).setScale(0, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double truncate(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.DOWN).doubleValue();

Review comment:
       I think it already does that. Can you point me to a case where this will fail?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r832151143



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TrigonometricTransformFunctions.java
##########
@@ -0,0 +1,281 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class TrigonometricTransformFunctions {
+  public static class Atan2TransformFunction extends BaseTransformFunction {
+    public static final String FUNCTION_NAME = "atan2";
+    private double[] _result;
+    private TransformFunction _leftTransformFunction;
+    private TransformFunction _rightTransformFunction;
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+      // Check that there are more than 1 arguments
+      if (arguments.size() != 2) {
+        throw new IllegalArgumentException("Exactly 2 arguments are required for Atan2 transform function");
+      }
+
+      _leftTransformFunction = arguments.get(0);
+      _rightTransformFunction = arguments.get(1);
+      Preconditions.checkArgument(
+          _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+              .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+    }
+
+    @Override
+    public TransformResultMetadata getResultMetadata() {
+      return DOUBLE_SV_NO_DICTIONARY_METADATA;
+    }
+
+    @Override
+    public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+      int length = projectionBlock.getNumDocs();
+
+      if (_result == null || _result.length < length) {
+        _result = new double[length];
+      }
+
+      double[] leftValues = _leftTransformFunction.transformToDoubleValuesSV(projectionBlock);
+      double[] rightValues = _rightTransformFunction.transformToDoubleValuesSV(projectionBlock);
+      for (int i = 0; i < length; i++) {
+        _result[i] = Math.atan2(leftValues[i], rightValues[i]);
+      }
+
+      return _result;
+    }
+  }
+
+  public static class DegreesTransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "degrees";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.toDegrees(values[i]);
+      }
+      ;
+    }
+  }
+
+  public static class AcosTransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "acos";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.acos(values[i]);
+      }
+      ;

Review comment:
       My bad. Missed in this file. Removed now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ccbba40) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **increase** coverage by `0.20%`.
   > The diff coverage is `77.16%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   + Coverage     70.76%   70.96%   +0.20%     
   - Complexity     4278     4283       +5     
   ============================================
     Files          1655     1661       +6     
     Lines         86665    87046     +381     
     Branches      13067    13134      +67     
   ============================================
   + Hits          61327    61772     +445     
   + Misses        21095    21012      -83     
   - Partials       4243     4262      +19     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.80% <14.18%> (+0.13%)` | :arrow_up: |
   | integration2 | `27.34% <14.18%> (-0.03%)` | :arrow_down: |
   | unittests1 | `67.07% <77.16%> (+0.09%)` | :arrow_up: |
   | unittests2 | `14.15% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `69.56% <37.50%> (+16.23%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `71.05% <64.28%> (-2.80%)` | :arrow_down: |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [44 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...ccbba40](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827350243



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }

Review comment:
       These functions without argument should be recognized as literal only expression in broker request handler, and replaced with the actual literal value, so in the query context it should already be overridden to literal




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828766021



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -57,12 +57,20 @@
   public LiteralTransformFunction(String literal) {
     _literal = literal;
     _dataType = inferLiteralDataType(literal);
+    double mathLiteral;
     if (_dataType.isNumeric()) {
       BigDecimal bigDecimal = new BigDecimal(_literal);
       _intLiteral = bigDecimal.intValue();
       _longLiteral = bigDecimal.longValue();
       _floatLiteral = bigDecimal.floatValue();
       _doubleLiteral = bigDecimal.doubleValue();
+    } else if ((mathLiteral = getMathConstant(literal)) != 0) {

Review comment:
       Ok. Yeah, I get for e.g. in a function like `startsWith(colA, 'E')` we should not be replacing `E` with Euler's constant.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827473390



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       Yeah, we don't need to strictly follow presto, and I just put it as a reference. Actually presto does not support `log`, postgres and sql server seems having opposite behavior, so let's pick one, or skip it for now




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r829106046



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in `TransformFunctionFactory` ensures this order.
   
   ```java
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r835231703



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       I would just use a flag rather than sentinel values. I don't know what the best thing to do is, but it probably isn't very important anyway. If this ever shows up as a bottleneck we can fix it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ccbba40) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **decrease** coverage by `0.83%`.
   > The diff coverage is `77.16%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   69.92%   -0.84%     
   - Complexity     4278     4283       +5     
   ============================================
     Files          1655     1661       +6     
     Lines         86665    87046     +381     
     Branches      13067    13134      +67     
   ============================================
   - Hits          61327    60871     -456     
   - Misses        21095    21934     +839     
   + Partials       4243     4241       -2     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.80% <14.18%> (+0.13%)` | :arrow_up: |
   | integration2 | `?` | |
   | unittests1 | `67.07% <77.16%> (+0.09%)` | :arrow_up: |
   | unittests2 | `14.15% <0.00%> (+<0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `69.56% <37.50%> (+16.23%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `71.05% <64.28%> (-2.80%)` | :arrow_down: |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [118 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...ccbba40](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825792598



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       Because of Tableau. Suppose a file as date time strings, but some of those values are NULL. since pinot doesn't support NULL, we store them as empty strings. 
   When someone applies a date transform function on such a column, an error is thrown. 
   
   However, the behavious should be a default value returned in those cases (or an explicit NULL value)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r836686912



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {

Review comment:
       ^^

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
##########
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+//TODO: The function should ideally be named 'round'
+// but it is not possible because of existing DateTimeFunction with same name.
+public class RoundDecimalTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "roundDecimal";
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private int _scale;
+  private boolean _hasLiteralArg;

Review comment:
       (minor)
   ```suggestion
     private boolean _fixedScale;
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -95,13 +97,55 @@ public static double exp(double a) {
     return Math.exp(a);
   }
 
-  @ScalarFunction
+  @ScalarFunction(names = {"ln", "log"})
   public static double ln(double a) {
     return Math.log(a);
   }
 
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
   @ScalarFunction
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction

Review comment:
       To match presto
   ```suggestion
     @ScalarFunction(names = {"pow", "power"})
   ```

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private double _exponent;
+  private boolean _hasLiteralArg;

Review comment:
       (minor)
   ```suggestion
     private boolean _fixedExponent;
   ```

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
##########
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+//TODO: The function should ideally be named 'round'
+// but it is not possible because of existing DateTimeFunction with same name.
+public class RoundDecimalTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "roundDecimal";
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private int _scale;
+  private boolean _hasLiteralArg;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    int numArguments = arguments.size();
+    // Check that there are more than 2 arguments or no arguments
+    if (numArguments < 1 || numArguments > 2) {
+      throw new IllegalArgumentException(
+          "roundDecimal transform function supports either 1 or 2 arguments. Num arguments provided: " + numArguments);
+    }
+
+    _hasLiteralArg = false;
+    _leftTransformFunction = arguments.get(0);
+    if (numArguments > 1) {
+      _rightTransformFunction = arguments.get(1);
+      if (_rightTransformFunction instanceof LiteralTransformFunction) {
+        _scale = Integer.parseInt(((LiteralTransformFunction) _rightTransformFunction).getLiteral());
+        _hasLiteralArg = true;
+      }
+      Preconditions.checkArgument(
+          _rightTransformFunction.getResultMetadata().isSingleValue() && isIntegralResultDatatype(
+              _rightTransformFunction),
+          "Argument must be single-valued with type INT or LONG for transform function: %s", getName());
+    } else {
+      _rightTransformFunction = null;

Review comment:
       Can simplify the handling by setting `_scale = 0` and `_fixedScale = true`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double roundDecimal(double a) {
+    return BigDecimal.valueOf(a).setScale(0, RoundingMode.HALF_UP).doubleValue();

Review comment:
       ^^




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r820497946



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }

Review comment:
       Constants should be implemented with `LiteralTransformFunction` so they can be shared between segments.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r831090957



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -164,7 +164,7 @@ protected void applyMathOperator(double[] values, int length) {
 
   public static class Log2TransformFunction extends SingleParamMathTransformFunction {
     public static final String FUNCTION_NAME = "log2";
-    public static final double LOG_BASE = Math.log(2);
+    public static final double LOG_BASE = 0.6931471805599453d;

Review comment:
       This isn't necessary, the field is already `static final` so is trusted as a constant




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (59f40af) into [master](https://codecov.io/gh/apache/pinot/commit/f67b5aacc5b494a2f5d78e87d67a84ea3aadc99a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f67b5aa) will **decrease** coverage by `43.43%`.
   > The diff coverage is `14.64%`.
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8304       +/-   ##
   =============================================
   - Coverage     70.76%   27.33%   -43.44%     
   =============================================
     Files          1631     1632        +1     
     Lines         85490    85840      +350     
     Branches      12878    12932       +54     
   =============================================
   - Hits          60499    23461    -37038     
   - Misses        20819    60197    +39378     
   + Partials       4172     2182     -1990     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `27.33% <14.64%> (-0.19%)` | :arrow_down: |
   | unittests1 | `?` | |
   | unittests2 | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `40.00% <0.00%> (-60.00%)` | :arrow_down: |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `17.39% <0.00%> (-35.95%)` | :arrow_down: |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `4.76% <0.00%> (-91.16%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (-95.66%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `13.51% <7.14%> (-59.51%)` | :arrow_down: |
   | ... and [1210 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f67b5aa...59f40af](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r831080942



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TruncateDecimalTransformFunction.java
##########
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+public class TruncateDecimalTransformFunction extends BaseTransformFunction {

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826407640



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {

Review comment:
       I see, let's keep the start index as 0.
   I'd suggest adding it as `splitPart` so that it does not collide with `split` with limit. We can also add a `split` with 3 arguments which returns `StringUtils.split(input, delimiter, limit)`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826404506



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       #8310 addressed the null time issue. With this PR, do we still need to handle empty string values?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r822041725



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @richardstartin Agree that `ScalarFunction` is not as efficient as `TransformFunction`, but we need `ScalarFunction` for ingestion transform and post-aggregation. We can add `ScalarFunction` for functionality first, then add `TransformFunction` for performance optimization later.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r822011259



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }
+
+  @ScalarFunction(names = {"round_decimal"})

Review comment:
       No need to use this alias. Both `round_decimal` and `roundDecimal` will be canonicalized to `rounddecimal`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -180,6 +180,24 @@ public static String rtrim(String input) {
     return RTRIM.matcher(input).replaceAll("");
   }
 
+  /**
+   * @param input
+   * @return trim spaces from right side of the string
+   */
+  @ScalarFunction
+  public static String leftSubStr(String input, int length) {
+    return StringUtils.left(input, length);
+  }
+
+  /**
+   * @param input
+   * @return trim spaces from right side of the string

Review comment:
       Fix the javadoc

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -715,4 +730,41 @@ public static String dateTimeConvert(String timeValueStr, String inputFormatStr,
       return new DateTimeFormatSpec(outputFormatStr).fromMillisToFormat(roundedTimeValueMs);
     }
   }
+
+  @ScalarFunction(names = {"timestampAdd", "dateAdd"})

Review comment:
       Please add some javadoc for these 2 methods

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {
+    String[] splitString = StringUtils.split(input, delimiter);
+    if(index < splitString.length){
+      return splitString[index];
+    } else {
+      return "null";
+    }
+  }
+
+  /**
+   * see String#split(String)
+   * @param input
+   * @param times
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String repeat(String input, int times) {
+    return StringUtils.repeat(input, times);
+  }
+
+  /**
+   * see String#split(String)

Review comment:
       Update the javadoc

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {

Review comment:
       This is `splitPart` in presto (index starts with 1 though): https://prestodb.io/docs/current/functions/string.html
   Consider making them consistent

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       We can also add `log2`
   ```suggestion
     public static double log10(double a) {
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -715,4 +730,41 @@ public static String dateTimeConvert(String timeValueStr, String inputFormatStr,
       return new DateTimeFormatSpec(outputFormatStr).fromMillisToFormat(roundedTimeValueMs);
     }
   }
+
+  @ScalarFunction(names = {"timestampAdd", "dateAdd"})
+  public static long timestampAdd(String unit, int interval, long timestamp) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).add(timestamp, interval);
+    return millis;
+  }
+
+  @ScalarFunction(names = {"timestampDiff", "dateDiff"})
+  public static long timestampDiff(String unit, long timestamp1, long timestamp2) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).getDifferenceAsLong(timestamp1, timestamp2);

Review comment:
       The return is not `millis` but value in `unit`. Also, seems the argument is in wrong order
   ```suggestion
       return getTimestampField(chronology, unit).getDifferenceAsLong(timestamp2, timestamp1);
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -180,6 +180,24 @@ public static String rtrim(String input) {
     return RTRIM.matcher(input).replaceAll("");
   }
 
+  /**
+   * @param input
+   * @return trim spaces from right side of the string

Review comment:
       Fix the javadoc

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double asin(double a) {
+    return Math.asin(a);
+  }
+
+  @ScalarFunction
+  public static double acos(double a) {
+    return Math.acos(a);
+  }
+
+  @ScalarFunction
+  public static double atan(double a) {
+    return Math.atan(a);
+  }
+
+  @ScalarFunction
+  public static double atan2(double a, double b) {
+    return Math.atan2(a, b);
+  }
+
+  @ScalarFunction
+  public static double sinh(double a) {
+    return Math.sinh(a);
+  }
+
+  @ScalarFunction
+  public static double cosh(double a) {
+    return Math.cosh(a);
+  }
+
+  @ScalarFunction
+  public static double tanh(double a) {
+    return Math.tanh(a);
+  }
+
+  @ScalarFunction
+  public static double degrees(double a) {
+    return Math.toDegrees(a);
+  }
+
+  @ScalarFunction
+  public static double radian(double a) {

Review comment:
       ```suggestion
     public static double radians(double a) {
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {

Review comment:
       What is `cot`?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction

Review comment:
       ```suggestion
     @ScalarFunction(names = {"pow", "power"})
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -715,4 +730,41 @@ public static String dateTimeConvert(String timeValueStr, String inputFormatStr,
       return new DateTimeFormatSpec(outputFormatStr).fromMillisToFormat(roundedTimeValueMs);
     }
   }
+
+  @ScalarFunction(names = {"timestampAdd", "dateAdd"})
+  public static long timestampAdd(String unit, int interval, long timestamp) {

Review comment:
       Should we put `interval` as `long`?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       Any specific reason why we want to support empty string? IMO it should throw exception if the time value doesn't comply with the pattern

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)

Review comment:
       Update the javadoc

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -715,4 +730,41 @@ public static String dateTimeConvert(String timeValueStr, String inputFormatStr,
       return new DateTimeFormatSpec(outputFormatStr).fromMillisToFormat(roundedTimeValueMs);
     }
   }
+
+  @ScalarFunction(names = {"timestampAdd", "dateAdd"})
+  public static long timestampAdd(String unit, int interval, long timestamp) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).add(timestamp, interval);
+    return millis;
+  }
+
+  @ScalarFunction(names = {"timestampDiff", "dateDiff"})
+  public static long timestampDiff(String unit, long timestamp1, long timestamp2) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).getDifferenceAsLong(timestamp1, timestamp2);
+    return millis;
+  }
+
+  private static DateTimeField getTimestampField(ISOChronology chronology, String unit) {

Review comment:
       This is available in `DateTimeUtils`

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/StringFunctions.java
##########
@@ -392,6 +431,45 @@ public static String normalize(String input, String form) {
     return StringUtils.split(input, delimiter);
   }
 
+  /**
+   * see String#split(String)
+   * @param input
+   * @param delimiter
+   * @return splits string on specified delimiter and returns an array.
+   */
+  @ScalarFunction
+  public static String split(String input, String delimiter, int index) {
+    String[] splitString = StringUtils.split(input, delimiter);
+    if(index < splitString.length){
+      return splitString[index];
+    } else {
+      return "null";
+    }
+  }
+
+  /**
+   * see String#split(String)

Review comment:
       Update the javadoc

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }
+
+  @ScalarFunction(names = {"round_decimal"})

Review comment:
       Can you add a `TODO` stating this should be named as `round` but the name is taken in the `DateTimeFunctions`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828282851



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       removed now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r827471141



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -273,9 +274,23 @@ public static String toDateTime(long millis, String pattern, String timezoneId)
    */
   @ScalarFunction
   public static long fromDateTime(String dateTimeString, String pattern) {
+    if (StringUtils.isEmpty(dateTimeString)) {

Review comment:
       I think it will throw NPE when `created_at` is `null`. My point is that we probably should not handle this in this general time handling function. The caller should catch the exception and fill the proper default value. The default value in the schema is not available to this function.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828423153



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {

Review comment:
       I have gone with ln, log2 and log10 following presto. I can add a variant as well allowing base parameter.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e81d81c) into [master](https://codecov.io/gh/apache/pinot/commit/515f7a2f9588f6fa96ae3e14a83ef9264e96531d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (515f7a2) will **increase** coverage by `0.08%`.
   > The diff coverage is `77.24%`.
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   + Coverage     70.76%   70.84%   +0.08%     
   + Complexity     4278     4277       -1     
   ============================================
     Files          1655     1660       +5     
     Lines         86665    86946     +281     
     Branches      13067    13117      +50     
   ============================================
   + Hits          61327    61599     +272     
   + Misses        21095    21089       -6     
   - Partials       4243     4258      +15     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.67% <14.13%> (+<0.01%)` | :arrow_up: |
   | integration2 | `27.23% <14.13%> (-0.14%)` | :arrow_down: |
   | unittests1 | `67.03% <77.24%> (+0.05%)` | :arrow_up: |
   | unittests2 | `14.08% <0.00%> (-0.08%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `69.56% <37.50%> (+16.23%)` | :arrow_up: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `71.05% <64.28%> (-2.80%)` | :arrow_down: |
   | [...orm/function/TruncateDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJ1bmNhdGVEZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `71.79% <71.79%> (ø)` | |
   | [...nsform/function/RoundDecimalTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUm91bmREZWNpbWFsVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `72.50% <72.50%> (ø)` | |
   | [...tor/transform/function/PowerTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vUG93ZXJUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `80.76% <80.76%> (ø)` | |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `95.23% <83.33%> (-0.73%)` | :arrow_down: |
   | [...form/function/TrigonometricTransformFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJpZ29ub21ldHJpY1RyYW5zZm9ybUZ1bmN0aW9ucy5qYXZh) | `86.25% <86.25%> (ø)` | |
   | [...orm/function/SingleParamMathTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vU2luZ2xlUGFyYW1NYXRoVHJhbnNmb3JtRnVuY3Rpb24uamF2YQ==) | `95.23% <90.90%> (-0.42%)` | :arrow_down: |
   | ... and [32 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [515f7a2...e81d81c](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834989372



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;

Review comment:
       cool. I have modified these classes for now.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826694601



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       On it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825835310



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       I understand that it would be _nice_ to have these available for ingestion transforms, but that isn't actually a SQL interface and those functions don't need to have the names used in SQL (though for simplicity it would be ideal if they did match!), so if the goal of adding the functions at all is to increase SQL compliance I would prioritise usage in SQL, which means implementing TransformFunctions, given the huge performance gap. 
   
   Support for ingestion transform configs can be added later when it's possible to do it without making queries inefficient. Moreover, once something has been added, it can't be removed, so queries relying on `@ScalarFunction` will be difficult to optimise without major rework.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter commented on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a7be2c2) into [master](https://codecov.io/gh/apache/pinot/commit/f67b5aacc5b494a2f5d78e87d67a84ea3aadc99a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f67b5aa) will **decrease** coverage by `56.58%`.
   > The diff coverage is `11.40%`.
   
   > :exclamation: Current head a7be2c2 differs from pull request most recent head 16939f0. Consider uploading reports for the commit 16939f0 to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/pinot/pull/8304/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #8304       +/-   ##
   =============================================
   - Coverage     70.76%   14.17%   -56.59%     
   + Complexity     4244       81     -4163     
   =============================================
     Files          1631     1592       -39     
     Lines         85490    83973     -1517     
     Branches      12878    12717      -161     
   =============================================
   - Hits          60499    11907    -48592     
   - Misses        20819    71164    +50345     
   + Partials       4172      902     -3270     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `?` | |
   | integration2 | `?` | |
   | unittests1 | `?` | |
   | unittests2 | `14.17% <11.40%> (-0.04%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...roker/requesthandler/BaseBrokerRequestHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcmVxdWVzdGhhbmRsZXIvQmFzZUJyb2tlclJlcXVlc3RIYW5kbGVyLmphdmE=) | `23.79% <0.00%> (-48.06%)` | :arrow_down: |
   | [...va/org/apache/pinot/common/config/NettyConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL05ldHR5Q29uZmlnLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...pache/pinot/common/config/provider/TableCache.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL3Byb3ZpZGVyL1RhYmxlQ2FjaGUuamF2YQ==) | `0.00% <0.00%> (-83.86%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...ot/common/function/scalar/ArithmeticFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0FyaXRobWV0aWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (-53.34%)` | :arrow_down: |
   | [...inot/common/function/scalar/DateTimeFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0RhdGVUaW1lRnVuY3Rpb25zLmphdmE=) | `0.00% <0.00%> (-95.92%)` | :arrow_down: |
   | [.../pinot/common/function/scalar/StringFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1N0cmluZ0Z1bmN0aW9ucy5qYXZh) | `0.00% <0.00%> (-73.02%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | `0.00% <0.00%> (-49.39%)` | :arrow_down: |
   | ... and [1343 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f67b5aa...16939f0](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r825769575



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }

Review comment:
       Sure, it needs to be a function, but it should be implemented as or similar to LiteralTransformFunction so that the array of e or pi values can be stored in the query context and shared across segments. Having said that, these will be very rarely invoked functions so maybe this can be deferred until someone actually complains about them.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r826408408



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/TrigonometricFunctions.java
##########
@@ -0,0 +1,74 @@
+package org.apache.pinot.common.function.scalar;
+
+import org.apache.pinot.spi.annotations.ScalarFunction;
+
+
+public class TrigonometricFunctions {
+  private TrigonometricFunctions(){
+  }
+
+  @ScalarFunction
+  public static double sin(double a) {
+    return Math.sin(a);
+  }
+
+  @ScalarFunction
+  public static double cos(double a) {
+    return Math.cos(a);
+  }
+
+  @ScalarFunction
+  public static double tan(double a) {
+    return Math.tan(a);
+  }
+
+  @ScalarFunction
+  public static double cot(double a) {

Review comment:
       Then the implementation is wrong. Also seems it is not a common one as it is not included in the `Math` library?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1067332089


   @KKcorps There are 6 hidden conversations (github automatically collapsed them) unaddressed, please take a look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r831080360



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -161,4 +161,55 @@ protected void applyMathOperator(double[] values, int length) {
       }
     }
   }
+
+  public static class Log2TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log2";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log(values[i]) / Math.log(2);

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830174453



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];

Review comment:
       Can you size this based on the number of docs in the block please? It will often be much smaller than 10k.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] richardstartin commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830182233



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -161,4 +161,55 @@ protected void applyMathOperator(double[] values, int length) {
       }
     }
   }
+
+  public static class Log2TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log2";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log(values[i]) / Math.log(2);
+      }
+      ;
+    }
+  }
+
+  public static class Log10TransformFunction extends SingleParamMathTransformFunction {
+    public static final String FUNCTION_NAME = "log10";
+
+    @Override
+    public String getName() {
+      return FUNCTION_NAME;
+    }
+
+    @Override
+    protected void applyMathOperator(double[] values, int length) {
+      for (int i = 0; i < length; i++) {
+        _results[i] = Math.log10(values[i]);
+      }
+      ;

Review comment:
       there's a few stray semi-colons in this file, we should enhance the checkstyle rules to ban these.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834519689



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
##########
@@ -43,7 +43,14 @@
   EXP("exp"),
   FLOOR("floor"),
   LN("ln"),
+  LOG("log"),
   SQRT("sqrt"),
+  LOG2("log2"),

Review comment:
       (minor) put all log functions together

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
##########
@@ -43,7 +43,14 @@
   EXP("exp"),
   FLOOR("floor"),
   LN("ln"),
+  LOG("log"),
   SQRT("sqrt"),
+  LOG2("log2"),
+  LOG10("log10"),
+  SIGN("sign"),
+  ROUND_DECIMAL("round_decimal"),

Review comment:
       Use camel case without underscore for non filter function
   ```suggestion
     ROUNDDECIMAL("roundDecimal"),
   ```

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {

Review comment:
       (minor) Keep all log functions together

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double roundDecimal(double a) {
+    return BigDecimal.valueOf(a).setScale(0, RoundingMode.HALF_UP).doubleValue();

Review comment:
       Can we use Math.round() for this function as `BigDecimal` calculation is much more costly? If not, please add some comments explaining the reason. Same for the one with scale

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;

Review comment:
       You may use `_doubleValuesSV` from the `BaseTransformFunction`, same for other functions

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
##########
@@ -109,7 +116,25 @@
   ST_WITHIN("ST_Within"),
 
   // Geo indexing
-  GEOTOH3("geoToH3");
+  GEOTOH3("geoToH3"),
+
+
+  //Trigonometry
+  SIN("sin"),
+  COS("cos"),
+  TAN("tan"),
+  COT("cot"),
+  ASIN("asin"),
+  ACOS("acos"),
+  ATAN("atan"),
+  ATAN2("atan2"),
+  SINH("sinh"),
+  COSH("cosh"),
+  TANH("tanh"),
+  DEGREES("degrees"),
+  RADIANS("radians");
+

Review comment:
       (minor, format) Avoid consecutive empty lines

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       We may avoid the boxing/unboxing using primitive double and use `NaN` to represent not set, same for other functions

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double roundDecimal(double a) {
+    return BigDecimal.valueOf(a).setScale(0, RoundingMode.HALF_UP).doubleValue();
+  }
+
+  @ScalarFunction
+  public static double truncate(double a, int b) {
+    return BigDecimal.valueOf(a).setScale(b, RoundingMode.DOWN).doubleValue();

Review comment:
       Can this handle negative values correctly? Can we use `Math.floor` for it?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/TransformFunctionType.java
##########
@@ -109,7 +116,25 @@
   ST_WITHIN("ST_Within"),
 
   // Geo indexing
-  GEOTOH3("geoToH3");
+  GEOTOH3("geoToH3"),
+

Review comment:
       (minor, format) Avoid consecutive empty lines (we might want to add a checkstyle rule later)

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/SingleParamMathTransformFunction.java
##########
@@ -161,4 +162,53 @@ protected void applyMathOperator(double[] values, int length) {
       }
     }
   }
+
+  public static class Log2TransformFunction extends SingleParamMathTransformFunction {

Review comment:
       (minor) put log functions together

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +106,46 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log2(double a) {
+    return Math.log(a) / Math.log(2);
+  }
+
+  @ScalarFunction
+  public static double log10(double a) {
+    return Math.log10(a);
+  }
+
+  //TODO: The function should ideally be named 'round'
+  // but it is not possible because of existing DateTimeFunction with same name.
+  @ScalarFunction
+  public static double roundDecimal(double a, int b) {

Review comment:
       Rename the second parameter to `scale` for readability

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TransformFunctionFactory.java
##########
@@ -83,7 +97,17 @@ private TransformFunctionFactory() {
           put(canonicalize(TransformFunctionType.EXP.getName().toLowerCase()), ExpTransformFunction.class);
           put(canonicalize(TransformFunctionType.FLOOR.getName().toLowerCase()), FloorTransformFunction.class);
           put(canonicalize(TransformFunctionType.LN.getName().toLowerCase()), LnTransformFunction.class);
+          put(canonicalize(TransformFunctionType.LOG.getName().toLowerCase()), LnTransformFunction.class);
           put(canonicalize(TransformFunctionType.SQRT.getName().toLowerCase()), SqrtTransformFunction.class);
+          put(canonicalize(TransformFunctionType.LOG2.getName().toLowerCase()), Log2TransformFunction.class);

Review comment:
       (minor) put log functions together




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r834756965



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       Made the change for this class. Is it fine if I use Integer.MIN_VALUE for other classes or leaved the boxed implementation as it is for now? I would prefer the latter.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r835754532



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,86 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private Double _exponent;

Review comment:
       cool.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r837140416



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/RoundDecimalTransformFunction.java
##########
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+
+
+//TODO: The function should ideally be named 'round'
+// but it is not possible because of existing DateTimeFunction with same name.
+public class RoundDecimalTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "roundDecimal";
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+  private int _scale;
+  private boolean _hasLiteralArg;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    int numArguments = arguments.size();
+    // Check that there are more than 2 arguments or no arguments
+    if (numArguments < 1 || numArguments > 2) {
+      throw new IllegalArgumentException(
+          "roundDecimal transform function supports either 1 or 2 arguments. Num arguments provided: " + numArguments);
+    }
+
+    _hasLiteralArg = false;
+    _leftTransformFunction = arguments.get(0);
+    if (numArguments > 1) {
+      _rightTransformFunction = arguments.get(1);
+      if (_rightTransformFunction instanceof LiteralTransformFunction) {
+        _scale = Integer.parseInt(((LiteralTransformFunction) _rightTransformFunction).getLiteral());
+        _hasLiteralArg = true;
+      }
+      Preconditions.checkArgument(
+          _rightTransformFunction.getResultMetadata().isSingleValue() && isIntegralResultDatatype(
+              _rightTransformFunction),
+          "Argument must be single-valued with type INT or LONG for transform function: %s", getName());
+    } else {
+      _rightTransformFunction = null;

Review comment:
       The reason for doing it this way is to use optimal `Math.round` and `Math.floor` for this case instead  of relying on BigDecimal represenation.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828766021



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -57,12 +57,20 @@
   public LiteralTransformFunction(String literal) {
     _literal = literal;
     _dataType = inferLiteralDataType(literal);
+    double mathLiteral;
     if (_dataType.isNumeric()) {
       BigDecimal bigDecimal = new BigDecimal(_literal);
       _intLiteral = bigDecimal.intValue();
       _longLiteral = bigDecimal.longValue();
       _floatLiteral = bigDecimal.floatValue();
       _doubleLiteral = bigDecimal.doubleValue();
+    } else if ((mathLiteral = getMathConstant(literal)) != 0) {

Review comment:
       Ok. Yeah, I get for e.g. in a function like `startsWith(colA, 'E')` we should not be replacing `E` with Euler's constant.

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in TransformFunctionFactory ensures this order.
   
   ```
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in TransformFunctionFactory ensures this order.
   
   ```java
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       @mneedham I had discussion with @Jackie-Jiang in which he confirmed that Pinot prioritises TransformFunctions first and if not found, it will use ScalarFunctions. Both can have the same name. The following piece of code in `TransformFunctionFactory` ensures this order.
   
   ```java
     TransformFunction transformFunction;
           Class<? extends TransformFunction> transformFunctionClass = TRANSFORM_FUNCTION_MAP.get(functionName);
           if (transformFunctionClass != null) {
             // Transform function
             try {
               transformFunction = transformFunctionClass.newInstance();
             } catch (Exception e) {
               throw new RuntimeException("Caught exception while constructing transform function: " + functionName, e);
             }
           } else {
             // Scalar function
             FunctionInfo functionInfo = FunctionRegistry.getFunctionInfo(functionName, numArguments);
             if (functionInfo == null) {
               if (FunctionRegistry.containsFunction(functionName)) {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s with %d parameters", functionName, numArguments));
               } else {
                 throw new BadQueryRequestException(
                   String.format("Unsupported function: %s not found", functionName));
               }
             }
             transformFunction = new ScalarTransformFunctionWrapper(functionInfo);
           }
   ```
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r830250099



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/PowerTransformFunction.java
##########
@@ -0,0 +1,74 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+
+
+public class PowerTransformFunction extends BaseTransformFunction {
+  public static final String FUNCTION_NAME = "power";
+  private double[] _result;
+  private TransformFunction _leftTransformFunction;
+  private TransformFunction _rightTransformFunction;
+
+  @Override
+  public String getName() {
+    return FUNCTION_NAME;
+  }
+
+  @Override
+  public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+    // Check that there are more than 1 arguments
+    if (arguments.size() != 2) {
+      throw new IllegalArgumentException("Exactly 2 arguments are required for power transform function");
+    }
+
+    _leftTransformFunction = arguments.get(0);
+    _rightTransformFunction = arguments.get(1);
+    Preconditions.checkArgument(
+        _leftTransformFunction.getResultMetadata().isSingleValue() || _rightTransformFunction.getResultMetadata()
+            .isSingleValue(), "Argument must be single-valued for transform function: %s", getName());
+  }
+
+  @Override
+  public TransformResultMetadata getResultMetadata() {
+    return DOUBLE_SV_NO_DICTIONARY_METADATA;
+  }
+
+  @Override
+  public double[] transformToDoubleValuesSV(ProjectionBlock projectionBlock) {
+    if (_result == null) {
+      _result = new double[DocIdSetPlanNode.MAX_DOC_PER_CALL];

Review comment:
       Makes sense. But this is definetely the case with all the existing functions as well. I will make changes to these functions here and raise a new PR in future for other functions.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] codecov-commenter edited a comment on pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#issuecomment-1066731691


   # [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#8304](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3c4cde7) into [master](https://codecov.io/gh/apache/pinot/commit/f67b5aacc5b494a2f5d78e87d67a84ea3aadc99a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f67b5aa) will **decrease** coverage by `0.94%`.
   > The diff coverage is `72.29%`.
   
   > :exclamation: Current head 3c4cde7 differs from pull request most recent head 9bdacb2. Consider uploading reports for the commit 9bdacb2 to get more accurate results
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #8304      +/-   ##
   ============================================
   - Coverage     70.76%   69.82%   -0.95%     
   - Complexity     4244     4252       +8     
   ============================================
     Files          1631     1642      +11     
     Lines         85490    86049     +559     
     Branches      12878    12945      +67     
   ============================================
   - Hits          60499    60085     -414     
   - Misses        20819    21795     +976     
   + Partials       4172     4169       -3     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration1 | `28.84% <30.98%> (-0.03%)` | :arrow_down: |
   | integration2 | `?` | |
   | unittests1 | `67.00% <66.45%> (+0.03%)` | :arrow_up: |
   | unittests2 | `14.18% <9.59%> (-0.03%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...roker/requesthandler/GrpcBrokerRequestHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcmVxdWVzdGhhbmRsZXIvR3JwY0Jyb2tlclJlcXVlc3RIYW5kbGVyLmphdmE=) | `0.00% <0.00%> (-78.58%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
   | [...common/function/scalar/TrigonometricFunctions.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL1RyaWdvbm9tZXRyaWNGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...g/apache/pinot/common/utils/helix/HelixHelper.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvaGVsaXgvSGVsaXhIZWxwZXIuamF2YQ==) | `48.79% <0.00%> (-0.60%)` | :arrow_down: |
   | [.../pinot/core/operator/filter/NotFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvTm90RmlsdGVyT3BlcmF0b3IuamF2YQ==) | `66.66% <ø> (+11.11%)` | :arrow_up: |
   | [...ry/optimizer/statement/JsonStatementOptimizer.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9vcHRpbWl6ZXIvc3RhdGVtZW50L0pzb25TdGF0ZW1lbnRPcHRpbWl6ZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../main/java/org/apache/pinot/core/util/OsCheck.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL09zQ2hlY2suamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...andpush/SegmentGenerationAndPushTaskGenerator.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3Mvc2VnbWVudGdlbmVyYXRpb25hbmRwdXNoL1NlZ21lbnRHZW5lcmF0aW9uQW5kUHVzaFRhc2tHZW5lcmF0b3IuamF2YQ==) | `2.94% <0.00%> (+0.14%)` | :arrow_up: |
   | [...l/realtime/converter/RealtimeSegmentConverter.java](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9yZWFsdGltZS9jb252ZXJ0ZXIvUmVhbHRpbWVTZWdtZW50Q29udmVydGVyLmphdmE=) | `80.00% <ø> (+1.87%)` | :arrow_up: |
   | ... and [183 more](https://codecov.io/gh/apache/pinot/pull/8304/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f67b5aa...9bdacb2](https://codecov.io/gh/apache/pinot/pull/8304?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828433081



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DateTimeFunctions.java
##########
@@ -715,4 +730,41 @@ public static String dateTimeConvert(String timeValueStr, String inputFormatStr,
       return new DateTimeFormatSpec(outputFormatStr).fromMillisToFormat(roundedTimeValueMs);
     }
   }
+
+  @ScalarFunction(names = {"timestampAdd", "dateAdd"})
+  public static long timestampAdd(String unit, int interval, long timestamp) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).add(timestamp, interval);
+    return millis;
+  }
+
+  @ScalarFunction(names = {"timestampDiff", "dateDiff"})
+  public static long timestampDiff(String unit, long timestamp1, long timestamp2) {
+    ISOChronology chronology = ISOChronology.getInstanceUTC();
+    long millis = getTimestampField(chronology, unit).getDifferenceAsLong(timestamp1, timestamp2);

Review comment:
       done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [pinot] KKcorps commented on a change in pull request #8304: Add commonly used math, string and date scalar functions in Pinot

Posted by GitBox <gi...@apache.org>.
KKcorps commented on a change in pull request #8304:
URL: https://github.com/apache/pinot/pull/8304#discussion_r828423933



##########
File path: pinot-common/src/main/java/org/apache/pinot/common/function/scalar/ArithmeticFunctions.java
##########
@@ -104,4 +111,30 @@ public static double ln(double a) {
   public static double sqrt(double a) {
     return Math.sqrt(a);
   }
+
+  @ScalarFunction
+  public static double sign(double a) {
+    return Math.signum(a);
+  }
+
+  @ScalarFunction
+  public static double pi() { return Math.PI; }
+
+  @ScalarFunction
+  public static double e() { return Math.E; }
+
+  @ScalarFunction
+  public static double power(double a, double b) {
+    return Math.pow(a, b);
+  }
+
+  @ScalarFunction
+  public static double log(double a) {
+    return Math.log10(a);
+  }

Review comment:
       Added transform functions classes for Arithmetic and Trigonometric functions. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org