You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/03 09:21:25 UTC

[GitHub] [spark] yaooqinn opened a new pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

yaooqinn opened a new pull request #28718:
URL: https://github.com/apache/spark/pull/28718


   
   
   ### What changes were proposed in this pull request?
   This PR is a followup for SPARK-31892, we want fully check for the formatter, which can warn us ahead if there is any behavior change when we update JDK.
   
   
   ### Why are the changes needed?
   
   test coverage and hotfix for silly mistakes in SPARK-31892
   
   
   ### Does this PR introduce _any_ user-facing change?
   <!--
   Note that it means *any* user-facing change including all aspects such as the documentation fix.
   If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
   If no, write 'No'.
   -->
   NO
   
   ### How was this patch tested?
   <!--
   If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
   If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
   If tests were not added, please describe why they were not added and/or why it was difficult to add.
   -->
   
   add tests
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638083899


   **[Test build #123479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123479/testReport)** for PR 28718 at commit [`6449e08`](https://github.com/apache/spark/commit/6449e08dcd03a36dc1f011f991c0245fa4135b08).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434737600



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting-invalid.sql
##########
@@ -0,0 +1,36 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS WITH INVALID PATTERNS ---
+
+-- separating this from datetime-formatting.sql ,because the text form
+-- for patterns with 5 letters in SimpleDateFormat varies from different JDKs
+select date_format('2018-11-17 13:33:33.333', 'GGGGG');
+-- pattern letter count can not be greater than 10
+select date_format('2018-11-17 13:33:33.333', 'yyyyyyyyyyy');
+select date_format('2018-11-17 13:33:33.333', 'YYYYYYYYYYY');
+-- q/L in JDK 8 will fail when the count is more than 2
+select date_format('2018-11-17 13:33:33.333', 'qqqqq');
+select date_format('2018-11-17 13:33:33.333', 'QQQQQ');
+select date_format('2018-11-17 13:33:33.333', 'MMMMM');
+select date_format('2018-11-17 13:33:33.333', 'LLLLL');
+select date_format('2018-11-17 13:33:33.333', 'www');
+select date_format('2018-11-17 13:33:33.333', 'WW');
+select date_format('2018-11-17 13:33:33.333', 'uuuuu');
+select date_format('2018-11-17 13:33:33.333', 'EEEEE');
+select date_format('2018-11-17 13:33:33.333', 'FF');
+select date_format('2018-11-17 13:33:33.333', 'ddd');
+-- DD is invalid in 8, but valid in 11 for the JDKs that PR builder uses

Review comment:
       DD is invalid if the day-of-year exceeds 100, but it becomes valid in Java 11.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638274689






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638084445






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638329784






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638329784






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638339186


   **[Test build #123493 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123493/testReport)** for PR 28718 at commit [`11c9653`](https://github.com/apache/spark/commit/11c96530a853d73a24e0c7a5a4c0fe0fe77a13f2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638258945


   **[Test build #123476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123476/testReport)** for PR 28718 at commit [`7e59d65`](https://github.com/apache/spark/commit/7e59d650f3006958b6ea48dca1c3b200a6bf9aad).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn closed pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
yaooqinn closed pull request #28718:
URL: https://github.com/apache/spark/pull/28718


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638510881






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638324128


   **[Test build #123490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123490/testReport)** for PR 28718 at commit [`83499a1`](https://github.com/apache/spark/commit/83499a19513462a21cbaf8b9e35f35cc9577afc2).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638076677


   **[Test build #123476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123476/testReport)** for PR 28718 at commit [`7e59d65`](https://github.com/apache/spark/commit/7e59d650f3006958b6ea48dca1c3b200a6bf9aad).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638364200


   **[Test build #123494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123494/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638087232


   **[Test build #123480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123480/testReport)** for PR 28718 at commit [`de63621`](https://github.com/apache/spark/commit/de63621fb392d4d9448d365c8d8bc6bee9f46012).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638212767


   **[Test build #123486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123486/testReport)** for PR 28718 at commit [`34aef79`](https://github.com/apache/spark/commit/34aef7957c2daa6f2577c59b1a8088d2942e49b3).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638324254


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123490/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638083899


   **[Test build #123479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123479/testReport)** for PR 28718 at commit [`6449e08`](https://github.com/apache/spark/commit/6449e08dcd03a36dc1f011f991c0245fa4135b08).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434546839



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime-legacy.sql.out
##########
@@ -996,3 +996,72 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 0003
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y-w-u YYYY-ww-uu YYY-W-uuu YY YYYYY uuuu E EE EEE EEEE') from ttt
+-- !query schema
+struct<date_format(t, Y-w-u YYYY-ww-uu YYY-W-uuu YY YYYYY uuuu E EE EEE EEEE):string>
+-- !query output
+1582-22-4 1582-22-04 1582-5-004 82 01582 0004 Thu Thu Thu Thursday
+1970-1-3 1970-01-03 1970-5-003 70 01970 0003 Wed Wed Wed Wednesday
+1970-53-4 1970-53-04 1970-5-004 70 01970 0004 Thu Thu Thu Thursday
+1996-13-7 1996-13-07 1996-4-007 96 01996 0007 Sun Sun Sun Sunday
+2018-46-6 2018-46-06 2018-3-006 18 02018 0006 Sat Sat Sat Saturday
+2020-1-2 2020-01-02 2020-5-002 20 02020 0002 Tue Tue Tue Tuesday
+2099-53-5 2099-53-05 2099-0-005 99 02099 0005 Fri Fri Fri Friday
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<>
+-- !query output
+java.lang.IllegalArgumentException
+Illegal pattern character 'q'
+
+
+-- !query
+select date_format(t, 'y-M-d H:m:s yyyy-MM-dd HH:mm:ss.SSS yy yyy yyyyy MMM MMMM L LL F h hh k kk K KK a') from ttt
+-- !query schema
+struct<date_format(t, y-M-d H:m:s yyyy-MM-dd HH:mm:ss.SSS yy yyy yyyyy MMM MMMM L LL F h hh k kk K KK a):string>
+-- !query output
+1582-5-31 19:40:35 1582-05-31 19:40:35.123 82 1582 01582 May May 5 05 5 7 07 19 19 7 07 PM
+1969-12-31 15:0:0 1969-12-31 15:00:00.000 69 1969 01969 Dec December 12 12 5 3 03 15 15 3 03 PM
+1970-12-31 4:59:59 1970-12-31 04:59:59.999 70 1970 01970 Dec December 12 12 5 4 04 4 04 4 04 AM
+1996-3-31 7:3:33 1996-03-31 07:03:33.123 96 1996 01996 Mar March 3 03 5 7 07 7 07 7 07 AM
+2018-11-17 5:33:33 2018-11-17 05:33:33.123 18 2018 02018 Nov November 11 11 3 5 05 5 05 5 05 AM
+2019-12-31 9:33:33 2019-12-31 09:33:33.123 19 2019 02019 Dec December 12 12 5 9 09 9 09 9 09 AM
+2100-1-1 1:33:33 2100-01-01 01:33:33.123 00 2100 02100 Jan January 1 01 1 1 01 1 01 1 01 AM
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt

Review comment:
       can we test `VV` separately? otherwise all other patterns listed here can't be tested with the legacy formatter.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638240705


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123479/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434548439



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       I'm not sure, but it seems like we should ban all the timezone letters during formatting. Spark's timestamp value doesn't have an associated timezone, it makes no sense to extract timezone from it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638232482






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434592867



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting.sql
##########
@@ -0,0 +1,19 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS ---
+
+-- valid formatter pattern check
+create temporary view ttt as select t from VALUES

Review comment:
       nit: 
   ```
   create temporary view v as select col from ...
   t(col)
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638231617


   **[Test build #123488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123488/testReport)** for PR 28718 at commit [`1e78a4e`](https://github.com/apache/spark/commit/1e78a4eb98211cc28ec52abfec84df0ba7fc7b04).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638482301


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123498/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434602604



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeFormatterHelperSuite.scala
##########
@@ -42,7 +46,7 @@ class DateTimeFormatterHelperSuite extends SparkFunSuite {
     }
     unsupportedLettersForParsing.foreach { l =>
       val e = intercept[IllegalArgumentException] {
-        convertIncompatiblePattern(s"$l", isParsing = true)
+        DateTimeFormatterHelper.convertIncompatiblePattern(s"$l", isParsing = true)

Review comment:
       Yes, the compiler is unhappy w/o this change.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638513280






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638232482






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638100797


   **[Test build #123482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123482/testReport)** for PR 28718 at commit [`3fea4de`](https://github.com/apache/spark/commit/3fea4def77afc42df69da7796dbfec1b4fe4be51).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638324243






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434548545



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       cc @MaxGekk 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638213536






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638226724


   **[Test build #123487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123487/testReport)** for PR 28718 at commit [`47d7322`](https://github.com/apache/spark/commit/47d7322d7122a928f79c8732e05793b4e2ee7d44).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638084445






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638294237






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434548439



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       I'm not sure, but it seems like we should ban all the timezone letters during formatting. Spark's timestamp value doesn't have an associated timezone, it makes no sense to extract timezone from it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638472943


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638240695






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434718052



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting-invalid.sql
##########
@@ -0,0 +1,43 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS WITH INVALID PATTERNS ---
+
+-- separating this from datetime-formatting.sql ,because the text form
+-- for patterns with 5 letters in SimpleDateFormat varies from different JDKs
+create temporary view v as select col from values
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') t(col);
+
+select col, date_format(col, 'GGGGG') from v;
+select col, date_format(col, 'yyyyyyyyyyy') from v; -- pattern letter count can not be greater than 10

Review comment:
       can we put the comment in a new line?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638472951


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123491/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638261740


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638409112






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638265935






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638273190


   **[Test build #123482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123482/testReport)** for PR 28718 at commit [`3fea4de`](https://github.com/apache/spark/commit/3fea4def77afc42df69da7796dbfec1b4fe4be51).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638212767


   **[Test build #123486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123486/testReport)** for PR 28718 at commit [`34aef79`](https://github.com/apache/spark/commit/34aef7957c2daa6f2577c59b1a8088d2942e49b3).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638408489


   **[Test build #123498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123498/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638261740






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638231617


   **[Test build #123488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123488/testReport)** for PR 28718 at commit [`1e78a4e`](https://github.com/apache/spark/commit/1e78a4eb98211cc28ec52abfec84df0ba7fc7b04).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638239161


   **[Test build #123479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123479/testReport)** for PR 28718 at commit [`6449e08`](https://github.com/apache/spark/commit/6449e08dcd03a36dc1f011f991c0245fa4135b08).
    * This patch **fails PySpark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638364200


   **[Test build #123494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123494/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638261640


   **[Test build #123487 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123487/testReport)** for PR 28718 at commit [`47d7322`](https://github.com/apache/spark/commit/47d7322d7122a928f79c8732e05793b4e2ee7d44).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638323833






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434593292



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting.sql
##########
@@ -0,0 +1,19 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS ---
+
+-- valid formatter pattern check
+create temporary view ttt as select t from VALUES

Review comment:
       and please lower case `VALUES`, or upper case all SQL keywords.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638328560


   **[Test build #123486 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123486/testReport)** for PR 28718 at commit [`34aef79`](https://github.com/apache/spark/commit/34aef7957c2daa6f2577c59b1a8088d2942e49b3).
    * This patch passes all tests.
    * This patch **does not merge cleanly**.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638512179


   **[Test build #123493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123493/testReport)** for PR 28718 at commit [`11c9653`](https://github.com/apache/spark/commit/11c96530a853d73a24e0c7a5a4c0fe0fe77a13f2).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638422065


   **[Test build #123488 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123488/testReport)** for PR 28718 at commit [`1e78a4e`](https://github.com/apache/spark/commit/1e78a4eb98211cc28ec52abfec84df0ba7fc7b04).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638408489


   **[Test build #123498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123498/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638101322






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638482294






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638077191






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638240695


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638274689






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638226724


   **[Test build #123487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123487/testReport)** for PR 28718 at commit [`47d7322`](https://github.com/apache/spark/commit/47d7322d7122a928f79c8732e05793b4e2ee7d44).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638413850


   Sorry for being late. Yes, @MaxGekk 's comment is correct.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638323332


   **[Test build #123491 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123491/testReport)** for PR 28718 at commit [`4d34f44`](https://github.com/apache/spark/commit/4d34f443691de60f28652a19a7d40cab7ba92307).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638323833






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638323332


   **[Test build #123491 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123491/testReport)** for PR 28718 at commit [`4d34f44`](https://github.com/apache/spark/commit/4d34f443691de60f28652a19a7d40cab7ba92307).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638265935






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638261751


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123487/
   Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] MaxGekk commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
MaxGekk commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638394597


   @cloud-fan See the example https://github.com/apache/spark/pull/28463 - add `[test-java11]`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638213536






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638423238






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638087801






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638077191






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434547591



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql
##########
@@ -168,3 +168,18 @@ select date_format(date '2018-11-17', 'yyyyyyyyyyy-MM-dd');
 -- SPARK-31879: the first day of week
 select date_format('2020-01-01', 'YYYY-MM-dd uu');
 select date_format('2020-01-01', 'YYYY-MM-dd uuuu');
+
+-- valid formatter pattern check
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t);
+select date_format(t, 'Y-w-u YYYY-ww-uu YYY-W-uuu YY YYYYY uuuu E EE EEE EEEE') from ttt;

Review comment:
       it's better to have a column to show the value of `t` as well.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638264269


   **[Test build #123480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123480/testReport)** for PR 28718 at commit [`de63621`](https://github.com/apache/spark/commit/de63621fb392d4d9448d365c8d8bc6bee9f46012).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638510881






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638087232


   **[Test build #123480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123480/testReport)** for PR 28718 at commit [`de63621`](https://github.com/apache/spark/commit/de63621fb392d4d9448d365c8d8bc6bee9f46012).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638087801






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638364904






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638364904






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638298956


   **[Test build #123490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123490/testReport)** for PR 28718 at commit [`83499a1`](https://github.com/apache/spark/commit/83499a19513462a21cbaf8b9e35f35cc9577afc2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638409112






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434549866



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       It seems to only display the value of `spark.sql.session.timeZone`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638513280






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638260521






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638472943






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] yaooqinn commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434468970



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       Do you know what causes this inaccuracy for dates before 1582-10-15？@cloud-fan




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638482294


   Merged build finished. Test FAILed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638472494


   **[Test build #123491 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123491/testReport)** for PR 28718 at commit [`4d34f44`](https://github.com/apache/spark/commit/4d34f443691de60f28652a19a7d40cab7ba92307).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638339894






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638482048


   **[Test build #123498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123498/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).
    * This patch **fails Spark unit tests**.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638298956


   **[Test build #123490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123490/testReport)** for PR 28718 at commit [`83499a1`](https://github.com/apache/spark/commit/83499a19513462a21cbaf8b9e35f35cc9577afc2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638227402






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434738032



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting.sql
##########
@@ -0,0 +1,74 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS ---
+
+create temporary view v as select col from values
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') t(col);
+
+select col, date_format(col, 'G GG GGG GGGG') from v;
+
+select col, date_format(col, 'y yy yyy yyyy yyyyy yyyyyy yyyyyyy yyyyyyyy yyyyyyyyy yyyyyyyyyy') from v;
+
+select col, date_format(col, 'Y YY YYY YYYY YYYYY YYYYYY YYYYYYY YYYYYYYY YYYYYYYYY YYYYYYYYYY') from v;
+
+select col, date_format(col, 'q qq') from v;
+
+select col, date_format(col, 'Q QQ QQQ QQQQ') from v;
+
+select col, date_format(col, 'M MM MMM MMMM') from v;
+
+select col, date_format(col, 'L LL') from v;
+
+select col, date_format(col, 'w ww') from v;
+
+select col, date_format(col, 'W') from v;
+
+select col, date_format(col, 'u uu uuu uuuu') from v;
+
+select col, date_format(col, 'E EE EEE EEEE') from v;
+
+select col, date_format(col, 'F') from v;
+
+select col, date_format(col, 'd dd') from v;
+
+select col, date_format(col, 'DD') from v where col = timestamp '2100-01-01 01:33:33.123America/Los_Angeles';
+select col, date_format(col, 'D DDD') from v;
+
+select col, date_format(col, 'H HH') from v;
+
+select col, date_format(col, 'h hh') from v;
+
+select col, date_format(col, 'k kk') from v;
+
+select col, date_format(col, 'K KK') from v;
+
+select col, date_format(col, 'm mm') from v;
+
+select col, date_format(col, 's ss') from v;
+
+select col, date_format(col, 'S SS SSS SSSS SSSSS SSSSSS SSSSSSS SSSSSSSS SSSSSSSSS') from v;
+
+-- add upper function here to avoid

Review comment:
       to avoid what?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434590780



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeFormatterHelperSuite.scala
##########
@@ -42,7 +46,7 @@ class DateTimeFormatterHelperSuite extends SparkFunSuite {
     }
     unsupportedLettersForParsing.foreach { l =>
       val e = intercept[IllegalArgumentException] {
-        convertIncompatiblePattern(s"$l", isParsing = true)
+        DateTimeFormatterHelper.convertIncompatiblePattern(s"$l", isParsing = true)

Review comment:
       is this change needed?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638076677


   **[Test build #123476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123476/testReport)** for PR 28718 at commit [`7e59d65`](https://github.com/apache/spark/commit/7e59d650f3006958b6ea48dca1c3b200a6bf9aad).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638339186


   **[Test build #123493 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123493/testReport)** for PR 28718 at commit [`11c9653`](https://github.com/apache/spark/commit/11c96530a853d73a24e0c7a5a4c0fe0fe77a13f2).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638101322






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434737132



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting-invalid.sql
##########
@@ -0,0 +1,36 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS WITH INVALID PATTERNS ---
+
+-- separating this from datetime-formatting.sql ,because the text form

Review comment:
       `.sql ,because` -> `.sql, because`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638260521






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638509070


   **[Test build #123494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123494/testReport)** for PR 28718 at commit [`eeceba5`](https://github.com/apache/spark/commit/eeceba5156a55f886525c4f0f3e3503885747169).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638407868


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434717715



##########
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-formatting-invalid.sql
##########
@@ -0,0 +1,43 @@
+--- TESTS FOR DATETIME FORMATTING FUNCTIONS WITH INVALID PATTERNS ---
+
+-- separating this from datetime-formatting.sql ,because the text form
+-- for patterns with 5 letters in SimpleDateFormat varies from different JDKs
+create temporary view v as select col from values
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') t(col);
+
+select col, date_format(col, 'GGGGG') from v;

Review comment:
       for invalid testing, we can just use literals?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638227402






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434542000



##########
File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeFormatterHelperSuite.scala
##########
@@ -23,43 +23,49 @@ import org.apache.spark.sql.catalyst.util.DateTimeFormatterHelper._
 class DateTimeFormatterHelperSuite extends SparkFunSuite {
 
   test("check incompatible pattern") {
-    assert(convertIncompatiblePattern("MM-DD-u") === "MM-DD-e")
-    assert(convertIncompatiblePattern("yyyy-MM-dd'T'HH:mm:ss.SSSz")
-      === "uuuu-MM-dd'T'HH:mm:ss.SSSz")
-    assert(convertIncompatiblePattern("yyyy-MM'y contains in quoted text'HH:mm:ss")
-      === "uuuu-MM'y contains in quoted text'HH:mm:ss")
-    assert(convertIncompatiblePattern("yyyy-MM-dd-u'T'HH:mm:ss.SSSz")
+    assert(convertIncompatiblePattern("MM-DD-u", isParsing = false) === "MM-DD-e")

Review comment:
       In this test suite, how about we add
   ```
   def convertIncompatiblePattern(pattern: String): String = {
     convertIncompatiblePattern(pattern, isParsing = false)
   }
   ```
   To reduce the code diff?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638100797


   **[Test build #123482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123482/testReport)** for PR 28718 at commit [`3fea4de`](https://github.com/apache/spark/commit/3fea4def77afc42df69da7796dbfec1b4fe4be51).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638339894






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28718: [SPARK-31892][SQL][FOLLOWUP][test-java11] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638423238






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #28718:
URL: https://github.com/apache/spark/pull/28718#discussion_r434558722



##########
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##########
@@ -1013,3 +1013,110 @@ select date_format('2020-01-01', 'YYYY-MM-dd uuuu')
 struct<date_format(CAST(2020-01-01 AS TIMESTAMP), YYYY-MM-dd uuuu):string>
 -- !query output
 2020-01-01 Wednesday
+
+
+-- !query
+create temporary view ttt as select t from VALUES
+ (timestamp '1582-06-01 11:33:33.123UTC+080000'),
+ (timestamp '1970-01-01 00:00:00.000Europe/Paris'),
+ (timestamp '1970-12-31 23:59:59.999Asia/Srednekolymsk'),
+ (timestamp '1996-04-01 00:33:33.123Australia/Darwin'),
+ (timestamp '2018-11-17 13:33:33.123Z'),
+ (timestamp '2020-01-01 01:33:33.123Asia/Shanghai'),
+ (timestamp '2100-01-01 01:33:33.123America/Los_Angeles') tt(t)
+-- !query schema
+struct<>
+-- !query output
+
+
+
+-- !query
+select date_format(t, 'Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy') from ttt
+-- !query schema
+struct<date_format(t, Y YY YYYY YYYYY YYYYY y yy yyy yyyy yyyyy):string>
+-- !query output
+1582 82 1582 01582 01582 1582 82 1582 1582 01582
+1970 70 1970 01970 01970 1969 69 1969 1969 01969
+1970 70 1970 01970 01970 1970 70 1970 1970 01970
+1996 96 1996 01996 01996 1996 96 1996 1996 01996
+2018 18 2018 02018 02018 2018 18 2018 2018 02018
+2020 20 2020 02020 02020 2019 19 2019 2019 02019
+2099 99 2099 02099 02099 2100 00 2100 2100 02100
+
+
+-- !query
+select date_format(t, 'q qq Q QQ QQQ QQQQ') from ttt
+-- !query schema
+struct<date_format(t, q qq Q QQ QQQ QQQQ):string>
+-- !query output
+1 01 1 01 Q1 1st quarter
+1 01 1 01 Q1 1st quarter
+2 02 2 02 Q2 2nd quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+4 04 4 04 Q4 4th quarter
+
+
+-- !query
+select date_format(t, 'M MM MMM MMMM L LL') from ttt
+-- !query schema
+struct<date_format(t, M MM MMM MMMM L LL):string>
+-- !query output
+1 01 Jan January 1 01
+11 11 Nov November 11 11
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+12 12 Dec December 12 12
+3 03 Mar March 3 03
+5 05 May May 5 05
+
+
+-- !query
+select date_format(t, 'W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a') from ttt
+-- !query schema
+struct<date_format(t, W ww d dd DDD u uu uuu uuuu F E EE EEE EEEE a):string>
+-- !query output
+0 53 1 01 001 5 05 Fri Friday 1 Fri Fri Fri Friday AM
+3 46 17 17 321 6 06 Sat Saturday 3 Sat Sat Sat Saturday AM
+4 13 31 31 091 7 07 Sun Sunday 3 Sun Sun Sun Sunday AM
+5 01 31 31 365 2 02 Tue Tuesday 3 Tue Tue Tue Tuesday AM
+5 01 31 31 365 3 03 Wed Wednesday 3 Wed Wed Wed Wednesday PM
+5 22 31 31 151 1 01 Mon Monday 3 Mon Mon Mon Monday PM
+5 53 31 31 365 4 04 Thu Thursday 3 Thu Thu Thu Thursday AM
+
+
+-- !query
+select date_format(t, 'h hh H HH k kk K KK m mm s ss SSS') from ttt
+-- !query schema
+struct<date_format(t, h hh H HH k kk K KK m mm s ss SSS):string>
+-- !query output
+1 01 1 01 1 01 1 01 33 33 33 33 123
+3 03 15 15 15 15 3 03 0 00 0 00 000
+4 04 4 04 4 04 4 04 59 59 59 59 999
+5 05 5 05 5 05 5 05 33 33 33 33 123
+7 07 19 19 19 19 7 07 40 40 35 35 123
+7 07 7 07 7 07 7 07 3 03 33 33 123
+9 09 9 09 9 09 9 09 33 33 33 33 123
+
+
+-- !query
+select date_format(t, 'VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ') from ttt
+-- !query schema
+struct<date_format(t, VV z zz zzz zzzz O OOOO X XX XXX XXXX XXXXX x xx xxx xxxx xxxx xxxxx Z ZZ ZZZ ZZZZ ZZZZZ):string>
+-- !query output
+America/Los_Angeles PST PST PST Pacific Standard Time GMT-7:52:58 GMT-07:52:58 -0752 -0752 -07:52 -075258 -07:52:58 -0752 -0752 -07:52 -075258 -075258 -07:52:58 -0752 -0752 -0752 GMT-07:52:58 -07:52:58

Review comment:
       actually, this is expected. The Spark test uses timezone `America/Los_Angeles`, which is indeed GMT-7:52:58 long time before, see https://www.politico.com/story/2017/11/18/railroads-initiative-synchronizes-us-clocks-nov-18-1883-244935
   
   Spark's timestamp value doesn't have an associated timezone, so extracting timezone means showing the session local timezone.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for valid pattern …

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638294237






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #28718: [SPARK-31892][SQL][FOLLOWUP] Improve test coverage for datetime pattern with formatter functions

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #28718:
URL: https://github.com/apache/spark/pull/28718#issuecomment-638373086


   @dongjoon-hyun do you know how to trigger java 11 test in the PR builder?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org