You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bruce Robbins (Jira)" <ji...@apache.org> on 2022/05/13 22:08:00 UTC
[jira] [Commented] (SPARK-39184) ArrayIndexOutOfBoundsException for some date/time sequences in some time-zones
[ https://issues.apache.org/jira/browse/SPARK-39184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17536913#comment-17536913 ]
Bruce Robbins commented on SPARK-39184:
---------------------------------------
I will take a stab at fixing this when the fix for SPARK-37544 is merged (which is changing code in the same area).
> ArrayIndexOutOfBoundsException for some date/time sequences in some time-zones
> ------------------------------------------------------------------------------
>
> Key: SPARK-39184
> URL: https://issues.apache.org/jira/browse/SPARK-39184
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.1, 3.3.0, 3.4.0
> Reporter: Bruce Robbins
> Priority: Major
>
> The following query gets an {{ArrayIndexOutOfBoundsException}} when run from the {{America/Los_Angeles}} time-zone:
> {noformat}
> spark-sql> select sequence(timestamp'2022-03-13 00:00:00', timestamp'2022-03-16 03:00:00', interval 1 day 1 hour) as x;
> 22/05/13 14:47:27 ERROR SparkSQLDriver: Failed in [select sequence(timestamp'2022-03-13 00:00:00', timestamp'2022-03-16 03:00:00', interval 1 day 1 hour) as x]
> java.lang.ArrayIndexOutOfBoundsException: 3
> {noformat}
> In fact, any such query will get an {{ArrayIndexOutOfBoundsException}} if the start-stop period in your time-zone includes more instances of "spring forward" than instances of "fall back" and the start-stop period is evenly divisible by the interval.
> In the {{America/Los_Angeles}} time-zone, examples include:
> {noformat}
> -- This query encompasses 2 instances of "spring forward" but only one
> -- instance of "fall back".
> select sequence(
> timestamp'2022-03-13',
> timestamp'2022-03-13' + (interval '42' hours * 209),
> interval '42' hours) as x;
> {noformat}
> {noformat}
> select sequence(
> timestamp'2022-03-13',
> timestamp'2022-03-13' + (interval '31' hours * 11),
> interval '31' hours) as x;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org