You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/07/14 08:01:48 UTC
[GitHub] [spark] beliefer opened a new pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
beliefer opened a new pull request #33341:
URL: https://github.com/apache/spark/pull/33341
### What changes were proposed in this pull request?
The current implement of `TimeWindow` only supports `TimestampType`. Spark added a new type `TimestampNTZType`, so we should support `TimestampNTZType` in expression `TimeWindow`.
### Why are the changes needed?
`TimestampNTZType` similar to `TimestampType`, we should support `TimestampNTZType` in expression `TimeWindow`.
### Does this PR introduce _any_ user-facing change?
'Yes'.
`TimeWindow` will accepts `TimestampNTZType`.
### How was this patch tested?
New tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881526637
**[Test build #141149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141149/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670491401
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -27,177 +29,234 @@ class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ def doTest(df: DataFrame): Unit = {
Review comment:
Nit: Shall we use a loop instead of defining functions..? e.g. `Seq(df1, df2).foreach...`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882499059
@gengliangwang @cloud-fan @dongjoon-hyun Thanks for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670986772
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -17,184 +17,249 @@
package org.apache.spark.sql
-import org.apache.spark.sql.catalyst.plans.logical.Expand
+import java.time.LocalDateTime
+
+import org.apache.spark.sql.catalyst.expressions.AttributeReference
+import org.apache.spark.sql.catalyst.plans.logical.{Aggregate, Expand}
import org.apache.spark.sql.functions._
import org.apache.spark.sql.test.SharedSparkSession
-import org.apache.spark.sql.types.StringType
+import org.apache.spark.sql.types._
class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ val df1 = Seq(("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
Review comment:
Yes. The string type will implict cast to the old timestamp type first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882233697
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880512910
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881348302
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882324142
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141217/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879780283
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880393625
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45564/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882499059
@gengliangwang @cloud-fan @dongjoon-hyun Thanks for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang closed pull request #33341:
URL: https://github.com/apache/spark/pull/33341
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881416190
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45662/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880512910
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882456708
Thanks, merging to master/3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879893561
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141007/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881932237
To @beliefer , please resolve the conflict.
To @gengliangwang , is this targeting Apache Spark 3.2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882456708
Thanks, merging to master/3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882006202
@dongjoon-hyun Yes currently all the TimestampNTZ changes are targeting Spark 3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880408615
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45564/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880031539
**[Test build #141017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141017/testReport)** for PR 33341 at commit [`a98cc86`](https://github.com/apache/spark/commit/a98cc86fe477c3be250b2cfa32897acdf6d22aa2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882324142
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141217/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670979148
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -17,184 +17,249 @@
package org.apache.spark.sql
-import org.apache.spark.sql.catalyst.plans.logical.Expand
+import java.time.LocalDateTime
+
+import org.apache.spark.sql.catalyst.expressions.AttributeReference
+import org.apache.spark.sql.catalyst.plans.logical.{Aggregate, Expand}
import org.apache.spark.sql.functions._
import org.apache.spark.sql.test.SharedSparkSession
-import org.apache.spark.sql.types.StringType
+import org.apache.spark.sql.types._
class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ val df1 = Seq(("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
Review comment:
if the column is string type, we now always cast it to the old timestamp type? or it depends on the config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879908797
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879739205
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881312313
> @beliefer could you rerun the GA? It seems not related though.
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879893561
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141007/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880408615
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45564/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r671132954
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -17,184 +17,249 @@
package org.apache.spark.sql
-import org.apache.spark.sql.catalyst.plans.logical.Expand
+import java.time.LocalDateTime
+
+import org.apache.spark.sql.catalyst.expressions.AttributeReference
+import org.apache.spark.sql.catalyst.plans.logical.{Aggregate, Expand}
import org.apache.spark.sql.functions._
import org.apache.spark.sql.test.SharedSparkSession
-import org.apache.spark.sql.types.StringType
+import org.apache.spark.sql.types._
class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ val df1 = Seq(("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
Review comment:
@cloud-fan from the current Type Coercion rules, when it comes to String and AnyTimestamp, we always cast as `AnyTimestampType.defaultConcreteType`. This is out of the scope of this PR. Let's revisit that later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881170478
**[Test build #141132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141132/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879704716
**[Test build #141007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141007/testReport)** for PR 33341 at commit [`b64802f`](https://github.com/apache/spark/commit/b64802f53c2790b7232c458214c8f048d49065fe).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879761639
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang closed pull request #33341:
URL: https://github.com/apache/spark/pull/33341
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881538936
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141149/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881351181
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141132/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880366199
**[Test build #141049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141049/testReport)** for PR 33341 at commit [`49315e7`](https://github.com/apache/spark/commit/49315e73532727193cf2c7f6ed1edf231ae4eae1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880382416
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45564/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670991708
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -17,184 +17,249 @@
package org.apache.spark.sql
-import org.apache.spark.sql.catalyst.plans.logical.Expand
+import java.time.LocalDateTime
+
+import org.apache.spark.sql.catalyst.expressions.AttributeReference
+import org.apache.spark.sql.catalyst.plans.logical.{Aggregate, Expand}
import org.apache.spark.sql.functions._
import org.apache.spark.sql.test.SharedSparkSession
-import org.apache.spark.sql.types.StringType
+import org.apache.spark.sql.types._
class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ val df1 = Seq(("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
Review comment:
is it expected? cc @gengliangwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882223359
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670924576
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -27,177 +29,234 @@ class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ def doTest(df: DataFrame): Unit = {
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880037916
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141017/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879825157
**[Test build #141017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141017/testReport)** for PR 33341 at commit [`a98cc86`](https://github.com/apache/spark/commit/a98cc86fe477c3be250b2cfa32897acdf6d22aa2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879881157
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882313763
**[Test build #141217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141217/testReport)** for PR 33341 at commit [`cc3543e`](https://github.com/apache/spark/commit/cc3543e29cc29c72d1fece7d5104e0c02f87ebd7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `public class ShuffleChecksumHelper `
* `class MutableCheckedOutputStream(out: OutputStream) extends OutputStream `
* `case class ShuffleChecksumBlockId(shuffleId: Int, mapId: Long, reduceId: Int) extends BlockId `
* `case class SessionWindow(timeColumn: Expression, gapDuration: Long) extends UnaryExpression`
* `case class LocalTimestamp(timeZoneId: Option[String] = None) extends LeafExpression`
* `class MergingSortWithSessionWindowStateIterator(`
* `case class SessionWindowStateStoreRestoreExec(`
* `case class SessionWindowStateStoreSaveExec(`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881351181
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141132/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881330793
**[Test build #141132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141132/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879780283
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45521/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879825157
**[Test build #141017 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141017/testReport)** for PR 33341 at commit [`a98cc86`](https://github.com/apache/spark/commit/a98cc86fe477c3be250b2cfa32897acdf6d22aa2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881416190
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45662/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880366199
**[Test build #141049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141049/testReport)** for PR 33341 at commit [`49315e7`](https://github.com/apache/spark/commit/49315e73532727193cf2c7f6ed1edf231ae4eae1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881380576
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45662/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670495627
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -27,177 +29,234 @@ class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ def doTest(df: DataFrame): Unit = {
+ checkAnswer(
+ df.groupBy(window($"time", "10 seconds"))
+ .agg(count("*").as("counts"))
+ .orderBy($"window.start".asc)
+ .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
Review comment:
We should have a test case for checking the result data type since it is not always TimestampType now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881352333
**[Test build #141149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141149/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880508957
ping @gengliangwang @cloud-fan @MaxGekk
cc @HyukjinKwon @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881538936
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141149/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r671723488
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -352,4 +463,31 @@ class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
)
}
}
+
+ test("SPARK-36091: Support TimestampNTZ type in expression TimeWindow") {
+ val df1 = Seq(
+ ("2016-03-27 19:39:30", 1, "a"),
+ ("2016-03-27 19:39:25", 2, "a")).toDF("time", "value", "id")
+ val df2 = Seq((LocalDateTime.parse("2016-03-27T19:39:30"), 1, "a"),
+ (LocalDateTime.parse("2016-03-27T19:39:25"), 2, "a")).toDF("time", "value", "id")
+ val type1 = StructType(
+ Seq(StructField("start", TimestampType), StructField("end", TimestampType)))
+ val type2 = StructType(
+ Seq(StructField("start", TimestampNTZType), StructField("end", TimestampNTZType)))
+
+ Seq((df1, type1), (df2, type2)).foreach { tuple =>
Review comment:
indentation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881242399
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45643/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881397886
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45662/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #33341:
URL: https://github.com/apache/spark/pull/33341#discussion_r670953749
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala
##########
@@ -27,177 +29,234 @@ class DataFrameTimeWindowingSuite extends QueryTest with SharedSparkSession {
import testImplicits._
test("simple tumbling window with record at window start") {
- val df = Seq(
- ("2016-03-27 19:39:30", 1, "a")).toDF("time", "value", "id")
-
- checkAnswer(
- df.groupBy(window($"time", "10 seconds"))
- .agg(count("*").as("counts"))
- .orderBy($"window.start".asc)
- .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
- Seq(
- Row("2016-03-27 19:39:30", "2016-03-27 19:39:40", 1)
+ def doTest(df: DataFrame): Unit = {
+ checkAnswer(
+ df.groupBy(window($"time", "10 seconds"))
+ .agg(count("*").as("counts"))
+ .orderBy($"window.start".asc)
+ .select($"window.start".cast("string"), $"window.end".cast("string"), $"counts"),
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881352333
**[Test build #141149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141149/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881309763
@beliefer could you rerun the GA? It seems not related though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882191281
**[Test build #141217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141217/testReport)** for PR 33341 at commit [`cc3543e`](https://github.com/apache/spark/commit/cc3543e29cc29c72d1fece7d5104e0c02f87ebd7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882233697
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880507565
**[Test build #141049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141049/testReport)** for PR 33341 at commit [`49315e7`](https://github.com/apache/spark/commit/49315e73532727193cf2c7f6ed1edf231ae4eae1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang closed pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
gengliangwang closed pull request #33341:
URL: https://github.com/apache/spark/pull/33341
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879908830
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881242399
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45643/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881229757
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45643/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879704716
**[Test build #141007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141007/testReport)** for PR 33341 at commit [`b64802f`](https://github.com/apache/spark/commit/b64802f53c2790b7232c458214c8f048d49065fe).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881213465
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45643/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879870935
**[Test build #141007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141007/testReport)** for PR 33341 at commit [`b64802f`](https://github.com/apache/spark/commit/b64802f53c2790b7232c458214c8f048d49065fe).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-879908830
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45532/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882191281
**[Test build #141217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141217/testReport)** for PR 33341 at commit [`cc3543e`](https://github.com/apache/spark/commit/cc3543e29cc29c72d1fece7d5104e0c02f87ebd7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-881170478
**[Test build #141132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141132/testReport)** for PR 33341 at commit [`2251cb0`](https://github.com/apache/spark/commit/2251cb0db8a722027f306548c1a5d3c81ca89202).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882499059
@gengliangwang @cloud-fan @dongjoon-hyun Thanks for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-882209959
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45730/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33341: [WIP][SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33341:
URL: https://github.com/apache/spark/pull/33341#issuecomment-880037916
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141017/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org