You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/13 09:51:32 UTC
[GitHub] [spark] Fokko opened a new pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Fokko opened a new pull request #28821:
URL: https://github.com/apache/spark/pull/28821
Currently, when you take an average of a Timestamp, you'll end up with a Double, representing the seconds since epoch. This is because of old Hive behavior.
I strongly believe that it is better to return a Timestamp.
Behaviour in Postgres:
```
root@8c4241b617ec:/# psql postgres postgres
psql (12.3 (Debian 12.3-1.pgdg100+1))
Type "help" for help.
postgres=# CREATE TABLE timestamp_demo (ts TIMESTAMP);
CREATE TABLE
postgres=# INSERT INTO timestamp_demo VALUES('2019-01-01 18:22:11');
INSERT 0 1
postgres=# INSERT INTO timestamp_demo VALUES('2018-01-01 18:22:11');
INSERT 0 1
postgres=# INSERT INTO timestamp_demo VALUES('2017-01-01 18:22:11');
INSERT 0 1
postgres=# SELECT AVG(ts) FROM timestamp_demo;
ERROR: function avg(timestamp without time zone) does not exist
LINE 1: SELECT AVG(ts) FROM timestamp_demo;
```
Behaviour in MySQL:
```
root@bab43a5731e8:/# mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.0.20 MySQL Community Server - GPL
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> CREATE TABLE timestamp_demo (ts TIMESTAMP);
Query OK, 0 rows affected (0.05 sec)
mysql> INSERT INTO timestamp_demo VALUES('2019-01-01 18:22:11');
Query OK, 1 row affected (0.01 sec)
mysql> INSERT INTO timestamp_demo VALUES('2018-01-01 18:22:11');
Query OK, 1 row affected (0.01 sec)
mysql> INSERT INTO timestamp_demo VALUES('2017-01-01 18:22:11');
Query OK, 1 row affected (0.01 sec)
mysql> SELECT AVG(ts) FROM timestamp_demo;
+---------------------+
| AVG(ts) |
+---------------------+
| 20180101182211.0000 |
+---------------------+
1 row in set (0.00 sec)
```
Which is a YYYYMMDDHHMMSS format in double.
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-653065512
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643904220
**[Test build #124024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124024/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r447634917
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestData.scala
##########
@@ -73,6 +74,17 @@ private[sql] trait SQLTestData { self =>
df
}
+ protected lazy val timestampData: DataFrame = {
+ val df = spark.sparkContext.parallelize(
+ TestDataTimestamp(new Timestamp(1420140300000L)) :: // 2015-01-01 20:25:00
+ TestDataTimestamp(new Timestamp(1320140300000L)) :: // 2011-11-01 10:38:20
+ TestDataTimestamp(new Timestamp(1520140300000L)) :: // 2018-03-04 06:11:40
+ TestDataTimestamp(new Timestamp(-1409632500000L)) :: // 1925-05-01 19:44:32
Review comment:
No, this wasn't intentional. I can add some fractions if you like.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644110363
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643600098
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643904434
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643865812
**[Test build #124024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124024/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r449775917
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
##########
@@ -624,7 +624,6 @@ object TypeCoercion {
// Hive lets you do aggregation of timestamps... for some reason
case Sum(e @ TimestampType()) => Sum(Cast(e, DoubleType))
- case Average(e @ TimestampType()) => Average(Cast(e, DoubleType))
Review comment:
When chaning the existing bheivour, we need to update the migration guide and might need to add a legacy config to keep the current behaivour.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #28821:
URL: https://github.com/apache/spark/pull/28821
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647842258
**[Test build #124369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124369/testReport)** for PR 28821 at commit [`bd99e28`](https://github.com/apache/spark/commit/bd99e286ce3c041a4fcf5295ee7ed64300150f32).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `public final class MapOutputCommitMessage `
* `sealed trait LogisticRegressionSummary extends ClassificationSummary `
* `class _ClassificationSummary(JavaWrapper):`
* `class _TrainingSummary(JavaWrapper):`
* `class _BinaryClassificationSummary(_ClassificationSummary):`
* `class LogisticRegressionSummary(_ClassificationSummary):`
* `class LogisticRegressionTrainingSummary(LogisticRegressionSummary, _TrainingSummary):`
* `class BinaryLogisticRegressionSummary(_BinaryClassificationSummary,`
* `trait TimestampFormatterHelper extends TimeZoneAwareExpression `
* `case class WidthBucket(`
* `trait PredicateHelper extends Logging `
* ` case class TimeFormatters(date: DateFormatter, timestamp: TimestampFormatter)`
* `case class CoalesceBucketsInSortMergeJoin(conf: SQLConf) extends Rule[SparkPlan] `
* `case class ProcessingTimeTrigger(intervalMs: Long) extends Trigger `
* `case class ContinuousTrigger(intervalMs: Long) extends Trigger `
* `class StateStoreConf(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647773582
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647773126
**[Test build #124369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124369/testReport)** for PR 28821 at commit [`bd99e28`](https://github.com/apache/spark/commit/bd99e286ce3c041a4fcf5295ee7ed64300150f32).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643904434
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644898043
**[Test build #124130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124130/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643719783
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r439842079
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestData.scala
##########
@@ -73,6 +74,17 @@ private[sql] trait SQLTestData { self =>
df
}
+ protected lazy val timestampData: DataFrame = {
+ val df = spark.sparkContext.parallelize(
+ TestDataTimestamp(new Timestamp(1420140300000L)) :: // 2015-01-01 20:25:00
+ TestDataTimestamp(new Timestamp(1320140300000L)) :: // 2011-11-01 10:38:20
+ TestDataTimestamp(new Timestamp(1520140300000L)) :: // 2018-03-04 06:11:40
+ TestDataTimestamp(new Timestamp(-1409632500000L)) :: // 1925-05-01 19:44:32
Review comment:
Do you set the fractional part of seconds to zeros intentionally?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652940574
Found the issue. `(sum / count).cast(resultType)` needs to be `(sum / count.cast(DecimalType.LongDecimal)).cast(resultType)` similar to the line above. The for the division operation, both the dividend and divisor requires to be a Fractional type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644037848
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652963764
**[Test build #124898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124898/testReport)** for PR 28821 at commit [`3772374`](https://github.com/apache/spark/commit/3772374ed915ff57045107aecff5e52e70719953).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644898162
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644898178
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124130/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643599966
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647773126
**[Test build #124369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124369/testReport)** for PR 28821 at commit [`bd99e28`](https://github.com/apache/spark/commit/bd99e286ce3c041a4fcf5295ee7ed64300150f32).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643728269
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r446017826
##########
File path: sql/core/src/test/resources/sql-tests/results/udf/udf-window.sql.out
##########
@@ -154,17 +154,17 @@ SELECT val_timestamp, udf(cate), avg(val_timestamp) OVER(PARTITION BY udf(cate)
RANGE BETWEEN CURRENT ROW AND interval 23 days 4 hours FOLLOWING) FROM testData
ORDER BY udf(cate), val_timestamp
-- !query schema
-struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(CAST(val_timestamp AS DOUBLE)) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):double>
+struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(val_timestamp) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):timestamp>
-- !query output
NULL NULL NULL
-2017-07-31 17:00:00 NULL 1.5015456E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-08-05 23:13:20 a 1.502E9
-2020-12-30 16:00:00 a 1.6093728E9
-2017-07-31 17:00:00 b 1.5022728E9
-2017-08-17 13:00:00 b 1.503E9
-2020-12-30 16:00:00 b 1.6093728E9
+2017-07-31 17:00:00 NULL 2017-07-31 17:00:00
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-08-05 23:13:20 a 2017-08-05 23:13:20
+2020-12-30 16:00:00 a 2020-12-30 16:00:00
+2017-07-31 17:00:00 b 2017-08-09 03:00:00
+2017-08-17 13:00:00 b 2017-08-17 13:00:00
+2020-12-30 16:00:00 b 2020-12-30 16:00:00
Review comment:
Hm .. so we allow timestamp types whereas other DBMSes disallow?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644037334
**[Test build #124049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124049/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-653065512
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644110363
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647842456
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652942355
**[Test build #124895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124895/testReport)** for PR 28821 at commit [`d2c49f3`](https://github.com/apache/spark/commit/d2c49f30482eca138096bbf940f39abed9c36547).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643720025
**[Test build #123996 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123996/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652942355
**[Test build #124895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124895/testReport)** for PR 28821 at commit [`d2c49f3`](https://github.com/apache/spark/commit/d2c49f30482eca138096bbf940f39abed9c36547).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643720129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643720129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644037848
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652962464
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643865812
**[Test build #124024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124024/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643931292
Hi, @Fokko . The last failure means that we need to regenerate the output of `ThriftServerQueryTestSuite.sql`. To proceed more, please update it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644843670
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644843670
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652964285
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643864583
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644898162
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647842456
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643720025
**[Test build #123996 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123996/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643728269
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643728274
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123996/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652962472
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124895/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652962464
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko edited a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko edited a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652940574
Found the issue.
```
(sum / count).cast(resultType)
```
Needs to be:
```
(sum / count.cast(DecimalType.LongDecimal)).cast(resultType)
````
Similar to the line above. For the division operation, both the dividend and divisor requires to be a Fractional type. Great to have such extensive tests :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643864583
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644109882
**[Test build #124049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124049/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643728218
**[Test build #123996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123996/testReport)** for PR 28821 at commit [`707b0cf`](https://github.com/apache/spark/commit/707b0cf949e2532429bdc62d7ef219fe98a0751e).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644841971
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647773582
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-707419504
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644037334
**[Test build #124049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124049/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643600098
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644929089
Thanks for the restart @dongjoon-hyun. Another error now; let me dive into it and get back to y'all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652942935
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko edited a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko edited a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652940574
Found the issue. `(sum / count).cast(resultType)` needs to be `(sum / count.cast(DecimalType.LongDecimal)).cast(resultType)` similar to the line above. The for the division operation, both the dividend and divisor requires to be a Fractional type. Great to have such extensive tests :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644843087
**[Test build #124130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124130/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644843087
**[Test build #124130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124130/testReport)** for PR 28821 at commit [`83be227`](https://github.com/apache/spark/commit/83be227b21faf4007df4475d75944afdb211ea84).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-653065042
**[Test build #124898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124898/testReport)** for PR 28821 at commit [`3772374`](https://github.com/apache/spark/commit/3772374ed915ff57045107aecff5e52e70719953).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643931292
Hi, @Fokko . The last failure means that we need to regenerate the output file of `ThriftServerQueryTestSuite.sql`. To proceed more, please update it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643599966
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652942935
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652962271
**[Test build #124895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124895/testReport)** for PR 28821 at commit [`d2c49f3`](https://github.com/apache/spark/commit/d2c49f30482eca138096bbf940f39abed9c36547).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-653065532
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124898/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652963764
**[Test build #124898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124898/testReport)** for PR 28821 at commit [`3772374`](https://github.com/apache/spark/commit/3772374ed915ff57045107aecff5e52e70719953).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-644110376
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124049/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643904439
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124024/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Fokko commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
Fokko commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r447563125
##########
File path: sql/core/src/test/resources/sql-tests/results/udf/udf-window.sql.out
##########
@@ -154,17 +154,17 @@ SELECT val_timestamp, udf(cate), avg(val_timestamp) OVER(PARTITION BY udf(cate)
RANGE BETWEEN CURRENT ROW AND interval 23 days 4 hours FOLLOWING) FROM testData
ORDER BY udf(cate), val_timestamp
-- !query schema
-struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(CAST(val_timestamp AS DOUBLE)) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):double>
+struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(val_timestamp) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):timestamp>
-- !query output
NULL NULL NULL
-2017-07-31 17:00:00 NULL 1.5015456E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-08-05 23:13:20 a 1.502E9
-2020-12-30 16:00:00 a 1.6093728E9
-2017-07-31 17:00:00 b 1.5022728E9
-2017-08-17 13:00:00 b 1.503E9
-2020-12-30 16:00:00 b 1.6093728E9
+2017-07-31 17:00:00 NULL 2017-07-31 17:00:00
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-08-05 23:13:20 a 2017-08-05 23:13:20
+2020-12-30 16:00:00 a 2020-12-30 16:00:00
+2017-07-31 17:00:00 b 2017-08-09 03:00:00
+2017-08-17 13:00:00 b 2017-08-17 13:00:00
+2020-12-30 16:00:00 b 2020-12-30 16:00:00
Review comment:
Well, there isn't a real consensus around other DBMS'es. Keeping it a Timestamp seems like something that you would expect. MySQL's behavior is much more awkward in my opinion. Spark needs to pave the path on this one :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-643864421
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-652964285
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #28821:
URL: https://github.com/apache/spark/pull/28821#discussion_r449775121
##########
File path: sql/core/src/test/resources/sql-tests/results/udf/udf-window.sql.out
##########
@@ -154,17 +154,17 @@ SELECT val_timestamp, udf(cate), avg(val_timestamp) OVER(PARTITION BY udf(cate)
RANGE BETWEEN CURRENT ROW AND interval 23 days 4 hours FOLLOWING) FROM testData
ORDER BY udf(cate), val_timestamp
-- !query schema
-struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(CAST(val_timestamp AS DOUBLE)) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):double>
+struct<val_timestamp:timestamp,CAST(udf(cast(cate as string)) AS STRING):string,avg(val_timestamp) OVER (PARTITION BY CAST(udf(cast(cate as string)) AS STRING) ORDER BY val_timestamp ASC NULLS FIRST RANGE BETWEEN CURRENT ROW AND INTERVAL '23 days 4 hours' FOLLOWING):timestamp>
-- !query output
NULL NULL NULL
-2017-07-31 17:00:00 NULL 1.5015456E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-07-31 17:00:00 a 1.5016970666666667E9
-2017-08-05 23:13:20 a 1.502E9
-2020-12-30 16:00:00 a 1.6093728E9
-2017-07-31 17:00:00 b 1.5022728E9
-2017-08-17 13:00:00 b 1.503E9
-2020-12-30 16:00:00 b 1.6093728E9
+2017-07-31 17:00:00 NULL 2017-07-31 17:00:00
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-07-31 17:00:00 a 2017-08-02 11:04:26.666666
+2017-08-05 23:13:20 a 2017-08-05 23:13:20
+2020-12-30 16:00:00 a 2020-12-30 16:00:00
+2017-07-31 17:00:00 b 2017-08-09 03:00:00
+2017-08-17 13:00:00 b 2017-08-17 13:00:00
+2020-12-30 16:00:00 b 2020-12-30 16:00:00
Review comment:
It seems Oracle cannot accept timestamp for average, too. Any other system supporting this behaivour?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28821:
URL: https://github.com/apache/spark/pull/28821#issuecomment-647842464
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124369/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org