You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/24 08:37:45 UTC

[GitHub] [spark] MaxGekk opened a new pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

MaxGekk opened a new pull request #31951:
URL: https://github.com/apache/spark/pull/31951


   ### What changes were proposed in this pull request?
   1. Add new expression `MultiplyDTInterval` which multiplies a `DayTimeIntervalType` expression by a `NumericType` expression including ByteType, ShortType, IntegerType, LongType, FloatType, DoubleType, DecimalType.
   2. Extend binary arithmetic rules to support `numeric * day-time interval` and `day-time interval * numeric`.
   
   ### Why are the changes needed?
   To conform the ANSI SQL standard which requires such operation over day-time intervals:
   <img width="667" alt="Screenshot 2021-03-22 at 16 33 16" src="https://user-images.githubusercontent.com/1580697/111997810-77d1eb80-8b2c-11eb-951d-e43911d9c5db.png">
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   By running new tests:
   ```
   $ build/sbt "test:testOnly *IntervalExpressionsSuite"
   $ build/sbt "test:testOnly *ColumnExpressionSuite"
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600844914



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       > Shall we turn float/double to Decimal and do the calculation?
   
   Decimal (and Java BigDecimal) doesn't have representation for `NaN`, `PositiveInfinity` and `NegativeInfinity`, see:
   ```scala
   scala> import org.apache.spark.sql.types.Decimal
   import org.apache.spark.sql.types.Decimal
   
   scala> Decimal(Double.NaN)
   java.lang.NumberFormatException
     at java.math.BigDecimal.<init>(BigDecimal.java:497)
   
   scala> Decimal(Float.PositiveInfinity)
   java.lang.NumberFormatException
     at java.math.BigDecimal.<init>(BigDecimal.java:497)
   
   scala> Decimal(Float.MinValue)
   res2: org.apache.spark.sql.types.Decimal = -340282346638528860000000000000000000000
   ``` 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806502624


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136495/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805772868


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41031/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805633707


   **[Test build #136447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136447/testReport)** for PR 31951 at commit [`58db4dc`](https://github.com/apache/spark/commit/58db4dcb2cc622679bac04b7089f821843649138).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806192758


   **[Test build #136481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136481/testReport)** for PR 31951 at commit [`b80b489`](https://github.com/apache/spark/commit/b80b4892d8e15818a2267bd5da10a621ee56150e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806394458


   **[Test build #136502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136502/testReport)** for PR 31951 at commit [`d844c54`](https://github.com/apache/spark/commit/d844c54db22aa862fa3a9240afc1b0d933f78a4e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806465714


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41087/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806429366


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41085/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600424368



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       Seems that we should fail here too not just round it to the max long value




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600554518



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       IMO, the interval value expression's return type is an interval and it defines an interval itself. The expression behavior should respect the interval term rather than the multiplier or divisor. According to the standard, we should define  and raise `interval field overflow` rather than numeric overflows when `the number of significant digits exceeds the implementation-defined maximum number of significant digits`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805772868


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41031/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806491359


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41087/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805747872


   @yaooqinn @cloud-fan Could you review this PR, please.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806352036


   **[Test build #136495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136495/testReport)** for PR 31951 at commit [`e16055d`](https://github.com/apache/spark/commit/e16055d61ae8d5515766625a4a5254f34d198614).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806429385


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41085/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806570198


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136500/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806568763


   **[Test build #136500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136500/testReport)** for PR 31951 at commit [`1a35356`](https://github.com/apache/spark/commit/1a353566d0121e1bceb0c63fe0b8dc707403b3dc).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806266650


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41065/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805633707


   **[Test build #136447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136447/testReport)** for PR 31951 at commit [`58db4dc`](https://github.com/apache/spark/commit/58db4dcb2cc622679bac04b7089f821843649138).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806633217


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136502/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806461752


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41084/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600875548



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       I have found the [roundToLong()](https://guava.dev/releases/19.0/api/docs/com/google/common/math/DoubleMath.html#roundToLong(double,%20java.math.RoundingMode)) from Guava satisfies our needs completely. I am going to invoke it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806587773


   **[Test build #136502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136502/testReport)** for PR 31951 at commit [`d844c54`](https://github.com/apache/spark/commit/d844c54db22aa862fa3a9240afc1b0d933f78a4e).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806370798


   **[Test build #136500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136500/testReport)** for PR 31951 at commit [`1a35356`](https://github.com/apache/spark/commit/1a353566d0121e1bceb0c63fe0b8dc707403b3dc).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806192758


   **[Test build #136481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136481/testReport)** for PR 31951 at commit [`b80b489`](https://github.com/apache/spark/commit/b80b4892d8e15818a2267bd5da10a621ee56150e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806490909


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41087/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805743567


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41031/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806570198


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136500/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806501271


   **[Test build #136495 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136495/testReport)** for PR 31951 at commit [`e16055d`](https://github.com/apache/spark/commit/e16055d61ae8d5515766625a4a5254f34d198614).
    * This patch **fails SparkR unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806266635


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41065/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806287333


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136481/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806263257


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41065/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600523616



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       Does the SQL standard require such behavior? From my point of view, we can map `Double.PositiveInfinity` to `Long.MaxValue` as multiple double values can map to the same long value. We just should document such behavior, and borrow the text from `java.lang.Math.round()`
   ```
        <li>If the argument is negative infinity or any value less than or
          equal to the value of {@code Long.MIN_VALUE}, the result is
          equal to the value of {@code Long.MIN_VALUE}.
        <li>If the argument is positive infinity or any value greater than or
          equal to the value of {@code Long.MAX_VALUE}, the result is
          equal to the value of {@code Long.MAX_VALUE}.</ul>
   ``` 
   @srielau @cloud-fan WDYT?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806276377


   **[Test build #136481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136481/testReport)** for PR 31951 at commit [`b80b489`](https://github.com/apache/spark/commit/b80b4892d8e15818a2267bd5da10a621ee56150e).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806352036


   **[Test build #136495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136495/testReport)** for PR 31951 at commit [`e16055d`](https://github.com/apache/spark/commit/e16055d61ae8d5515766625a4a5254f34d198614).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806394458


   **[Test build #136502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136502/testReport)** for PR 31951 at commit [`d844c54`](https://github.com/apache/spark/commit/d844c54db22aa862fa3a9240afc1b0d933f78a4e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806461721


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41084/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806438970


   GA passed. Merging to master.
   Thank you, @yaooqinn @cloud-fan for your review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806431699


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41084/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk closed pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
MaxGekk closed pull request #31951:
URL: https://github.com/apache/spark/pull/31951


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805869740


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136447/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31951:
URL: https://github.com/apache/spark/pull/31951#discussion_r600544067



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala
##########
@@ -295,3 +295,43 @@ case class MultiplyYMInterval(
 
   override def toString: String = s"($left * $right)"
 }
+
+// Multiply a day-time interval by a numeric
+case class MultiplyDTInterval(
+    interval: Expression,
+    num: Expression)
+  extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable {
+  override def left: Expression = interval
+  override def right: Expression = num
+
+  override def inputTypes: Seq[AbstractDataType] = Seq(DayTimeIntervalType, NumericType)
+  override def dataType: DataType = DayTimeIntervalType
+
+  @transient
+  private lazy val evalFunc: (Long, Any) => Any = right.dataType match {
+    case _: IntegralType => (micros: Long, num) =>
+      Math.multiplyExact(micros, num.asInstanceOf[Number].longValue())
+    case _: DecimalType => (micros: Long, num) =>
+      val decimalRes = ((new Decimal).set(micros) * num.asInstanceOf[Decimal]).toJavaBigDecimal
+      decimalRes.setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()
+    case _: FractionalType => (micros: Long, num) =>
+      Math.round(micros * num.asInstanceOf[Number].doubleValue())
+  }
+
+  override def nullSafeEval(interval: Any, num: Any): Any = {
+    evalFunc(interval.asInstanceOf[Long], num)
+  }
+
+  override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = right.dataType match {
+    case _: IntegralType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.multiplyExact($m, $n)")
+    case _: DecimalType =>
+      defineCodeGen(ctx, ev, (m, n) =>
+        s"((new Decimal()).set($m).$$times($n)).toJavaBigDecimal()" +
+        ".setScale(0, java.math.RoundingMode.HALF_UP).longValueExact()")
+    case _: FractionalType =>
+      defineCodeGen(ctx, ev, (m, n) => s"java.lang.Math.round($m * (double)$n)")

Review comment:
       I don't have a strong opinion here. Shall we turn float/double to Decimal and do the calculation?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806424337


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41085/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806461752


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41084/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805869740


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136447/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806502624


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136495/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806491359


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41087/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805867663


   **[Test build #136447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136447/testReport)** for PR 31951 at commit [`58db4dc`](https://github.com/apache/spark/commit/58db4dcb2cc622679bac04b7089f821843649138).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806370798


   **[Test build #136500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136500/testReport)** for PR 31951 at commit [`1a35356`](https://github.com/apache/spark/commit/1a353566d0121e1bceb0c63fe0b8dc707403b3dc).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806429385


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41085/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806633217


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136502/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806287333


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136481/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-806266650


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41065/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31951: [SPARK-34850][SQL] Support multiply a day-time interval by a numeric

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31951:
URL: https://github.com/apache/spark/pull/31951#issuecomment-805772852


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41031/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org