You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/03 20:44:13 UTC

[GitHub] [spark] MaxGekk opened a new pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

MaxGekk opened a new pull request #31729:
URL: https://github.com/apache/spark/pull/31729


   ### What changes were proposed in this pull request?
   TODO
   
   ### Why are the changes needed?
   TODO
   
   ### Does this PR introduce _any_ user-facing change?
   The PR extends existing functionality. So, users can parallelize instances of the `java.time.Duration` class and collect them back:
   
   ```Scala
   scala> val ds = Seq(java.time.Duration.ofDays(10)).toDS
   ds: org.apache.spark.sql.Dataset[java.time.Duration] = [value: daytimeinterval]
   
   scala> ds.collect
   res0: Array[java.time.Duration] = Array(PT240H)
   ```
   
   
   ### How was this patch tested?
   - Added a few tests to `CatalystTypeConvertersSuite` to check conversion from/to `java.time.Duration`.
   - Checking row encoding by new tests in `RowEncoderSuite`.
   - Making literals of `DayTimeIntervalType` are tested in `LiteralExpressionSuite`
   - Check collecting by `DatasetSuite` and `JavaDatasetSuite`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790332682


   > > what will we use for the year-month?
   > 
   > @yaooqinn I think of java.time.Period (https://docs.oracle.com/javase/8/docs/api/java/time/Period.html)
   
   Does it mean that we only use the period with year and month filed and fail for the day part?
   
   BTW, what is the behavior of Dataframe.show(), will it be something like `PT1H2H3S`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on a change in pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #31729:
URL: https://github.com/apache/spark/pull/31729#discussion_r587371931



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystTypeConvertersSuite.scala
##########
@@ -223,4 +223,32 @@ class CatalystTypeConvertersSuite extends SparkFunSuite with SQLHelper {
       }
     }
   }
+
+  test("SPARK-34605: converting java.time.Duration to DayTimeIntervalType") {
+    Seq(
+      "PT20.123456S",
+      "P2DT3H4M",
+      "P2D").foreach { intervalStr =>

Review comment:
       @cloud-fan I created durations using another `Duration` methods. Also added negative tests.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790561664


   Kubernetes integration test status failure
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40335/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan closed pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31729:
URL: https://github.com/apache/spark/pull/31729


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790561683


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40335/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790322170


   @cloud-fan @HyukjinKwon @dongjoon-hyun @viirya @maropu @yaooqinn Could you review this PR, please.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790721278


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135752/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31729:
URL: https://github.com/apache/spark/pull/31729#discussion_r587204018



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/LiteralExpressionSuite.scala
##########
@@ -349,4 +349,22 @@ class LiteralExpressionSuite extends SparkFunSuite with ExpressionEvalHelper {
       assert(literalStr === "2021-02-03 17:50:03.456")
     }
   }
+
+  test("SPARK-34605: construct literals from java.time.Duration") {
+    Seq(
+      Duration.ofNanos(0),
+      Duration.ofSeconds(-1),
+      Duration.ofNanos(123456000),
+      Duration.ofDays(106751991),
+      Duration.ofDays(-106751991)).foreach { duration =>
+      checkEvaluation(Literal(duration), duration)
+    }
+  }
+
+  test("SPARK-34605: construct literals from arrays of java.time.Duration") {
+    val duration0 = Duration.parse("P2DT3H4M")

Review comment:
       ditto




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31729:
URL: https://github.com/apache/spark/pull/31729#discussion_r587203926



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystTypeConvertersSuite.scala
##########
@@ -223,4 +223,32 @@ class CatalystTypeConvertersSuite extends SparkFunSuite with SQLHelper {
       }
     }
   }
+
+  test("SPARK-34605: converting java.time.Duration to DayTimeIntervalType") {
+    Seq(
+      "PT20.123456S",
+      "P2DT3H4M",
+      "P2D").foreach { intervalStr =>

Review comment:
       I think many people are not familiar with this format. Can we add comments or use a different way to create `Duration`?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790062534


   **[Test build #135721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135721/testReport)** for PR 31729 at commit [`b5b021e`](https://github.com/apache/spark/commit/b5b021e4d9df63045f60c36f067709e4e59058ac).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790767225


   thanks, merging to master!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790394112


   It's better we update the `sql-ref-datatypes.md` within this PR or create a separate JIRA to do later.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790326547


   I believe that `Duration` fits ANSI day-time interval well. But not related directly to this PR, what will we use for the year-month? 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790129981


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40303/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790349728


   > Does it mean that we only use the period with year and month filed ...
   
   Yes, we will use only year and month fields.
   
   > and fail for the day part?
   
   Let's discuss this in https://issues.apache.org/jira/browse/SPARK-34615
   
   > BTW, what is the behavior of Dataframe.show(), will it be something like PT1H2H3S
   
   No-no. It will be in the format that conforms to the ANSI SQL standard like:
   ```
   INTERVAL ‘12 45:20:59.123’ DAY TO SECOND
   ```
   
   BTW, the format `PT1H2H3S` is from the ISO-8601 standard.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790062534


   **[Test build #135721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135721/testReport)** for PR 31729 at commit [`b5b021e`](https://github.com/apache/spark/commit/b5b021e4d9df63045f60c36f067709e4e59058ac).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790561683


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40335/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790719732


   **[Test build #135752 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135752/testReport)** for PR 31729 at commit [`b826afd`](https://github.com/apache/spark/commit/b826afda86fd27a239730859cb5c7d4db952823d).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790222220


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135721/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790545855


   **[Test build #135752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135752/testReport)** for PR 31729 at commit [`b826afd`](https://github.com/apache/spark/commit/b826afda86fd27a239730859cb5c7d4db952823d).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790721278


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135752/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790220240


   **[Test build #135721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135721/testReport)** for PR 31729 at commit [`b5b021e`](https://github.com/apache/spark/commit/b5b021e4d9df63045f60c36f067709e4e59058ac).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790328539


   > what will we use for the year-month?
   
   @yaooqinn I think of java.time.Period (https://docs.oracle.com/javase/8/docs/api/java/time/Period.html)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790114713


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40303/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790557050


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40335/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790222220


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135721/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790529333


   > It's better we update the sql-ref-datatypes.md
   
   @yaooqinn Let's update docs separately. I opened the sub-task SPARK-34619 for that. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31729:
URL: https://github.com/apache/spark/pull/31729#discussion_r587205217



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/CatalystTypeConvertersSuite.scala
##########
@@ -223,4 +223,32 @@ class CatalystTypeConvertersSuite extends SparkFunSuite with SQLHelper {
       }
     }
   }
+
+  test("SPARK-34605: converting java.time.Duration to DayTimeIntervalType") {
+    Seq(
+      "PT20.123456S",
+      "P2DT3H4M",
+      "P2D").foreach { intervalStr =>

Review comment:
       Cam we add some negative cases?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #31729: [SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790545855


   **[Test build #135752 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135752/testReport)** for PR 31729 at commit [`b826afd`](https://github.com/apache/spark/commit/b826afda86fd27a239730859cb5c7d4db952823d).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790129981


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40303/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #31729: [WIP][SPARK-34605][SQL] Support `java.time.Duration` as an external type of the day-time interval type

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31729:
URL: https://github.com/apache/spark/pull/31729#issuecomment-790087665


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40303/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org