You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/13 03:53:27 UTC
[GitHub] [spark] TJX2014 opened a new pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
TJX2014 opened a new pull request #28819:
URL: https://github.com/apache/spark/pull/28819
### What changes were proposed in this pull request?
Add judge equal condition as bigger in `org.apache.spark.sql.catalyst.expressions.Sequence.TemporalSequenceImpl#eval`
Unit for interval `day`, `month`, `year`
### Why are the changes needed?
Bug exists when sequence input get a equal start and end date, which will occur `while loop` forever
### Does this PR introduce _any_ user-facing change?
Yes,
Before this PR, people will get a `java.lang.ArrayIndexOutOfBoundsException`, when eval as below:
`sql("select sequence(cast('2011-03-01' as date), cast('2011-03-01' as date), interval 1 year)").show(false)
`
### How was this patch tested?
Unit test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643563705
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643757784
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643758259
**[Test build #124007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124007/testReport)** for PR 28819 at commit [`c1765e1`](https://github.com/apache/spark/commit/c1765e1af8ee5de6b61261234c0e8db2ba299712).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643758444
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643740231
> I think it matches the behaviour with Presto's (see #21155). Shall we check how it works in Presto?
Sure, we can get result from Presto as follows：
`
**presto> select sequence(date('2011-03-01'),date('2011-03-01'),interval '1' year);
_col0
[2011-03-01]
(1 row)**
Query 20200614_091013_00013_wnnjv, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
presto> select sequence(date('2011-03-01'),date('2012-03-01'),interval '1' year);
_col0
[2011-03-01, 2012-03-01]
(1 row)
Query 20200614_091017_00014_wnnjv, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
presto> select sequence(date('2011-03-01'),date('2010-03-01'),interval '1' year);
Query 20200614_091021_00015_wnnjv failed: sequence stop value should be greater than or equal to start value if step is greater than zero otherwise stop should be less than or equal to start
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643791390
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] DaveDeCaprio commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
DaveDeCaprio commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643619880
I'm not an admin or committer, but this patch looks good to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643563821
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 edited a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 edited a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643740231
> I think it matches the behaviour with Presto's (see #21155). Shall we check how it works in Presto?
Sure, we can get result from Presto(presto-server-0.236) as follows：
`
**presto> select sequence(date('2011-03-01'),date('2011-03-01'),interval '1' year);
_col0
[2011-03-01]
(1 row)**
Query 20200614_091013_00013_wnnjv, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
presto> select sequence(date('2011-03-01'),date('2012-03-01'),interval '1' year);
_col0
[2011-03-01, 2012-03-01]
(1 row)
Query 20200614_091017_00014_wnnjv, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
presto> select sequence(date('2011-03-01'),date('2010-03-01'),interval '1' year);
Query 20200614_091021_00015_wnnjv failed: sequence stop value should be greater than or equal to start value if step is greater than zero otherwise stop should be less than or equal to start
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #28819:
URL: https://github.com/apache/spark/pull/28819
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643758259
**[Test build #124007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124007/testReport)** for PR 28819 at commit [`c1765e1`](https://github.com/apache/spark/commit/c1765e1af8ee5de6b61261234c0e8db2ba299712).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-646933713
@dongjoon-hyun Thanks, I am willing to, I still have a question, Could you please help me check.Could I base `master` to use `CalendarInterval.fromString` instead of `stringToInterval` or just base `branch-2.4` to use `CalendarInterval.fromString` but base `master` to use `stringToInterval`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643722914
I think it matches the behaviour with Presto's (see https://github.com/apache/spark/pull/21155). Shall we check how it works in Presto?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643563705
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643791390
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-646923541
Hi, @TJX2014 .
In branch-2.4, `stringToInterval` doesn't exist. Could you make a PR for Apache Spark 2.4.7 please?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643758444
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643791166
**[Test build #124007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124007/testReport)** for PR 28819 at commit [`c1765e1`](https://github.com/apache/spark/commit/c1765e1af8ee5de6b61261234c0e8db2ba299712).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-646934894
Thanks, I will make a PR for branch-2.4.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-643563821
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 removed a comment on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-646933713
@dongjoon-hyun Thanks, I am willing to, I still have a question, Could you please help me check.Could I base `master` to use `CalendarInterval.fromString` instead of `stringToInterval` or just base `branch-2.4` to use `CalendarInterval.fromString` but base `master` to use `stringToInterval`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] TJX2014 commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates
Posted by GitBox <gi...@apache.org>.
TJX2014 commented on pull request #28819:
URL: https://github.com/apache/spark/pull/28819#issuecomment-646968614
> Thanks, I will make a PR for branch-2.4.
@dongjoon-hyun This is the backport.[#28877](https://github.com/apache/spark/pull/28877)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org