You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by "deepujain (via GitHub)" <gi...@apache.org> on 2026/03/09 19:34:25 UTC

[PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

deepujain opened a new pull request, #54695:
URL: https://github.com/apache/spark/pull/54695

## Summary

In Spark 4.0.0 ANSI mode is on by default. The JIRA asks to minimize suggestions that tell users to turn ANSI off. The error `DATETIME_FIELD_OUT_OF_BOUNDS` previously had a subclass `WITH_SUGGESTION` that appended "If necessary set spark.sql.ansi.enabled to false to bypass this error." This PR removes that suggestion; users can use `try_*` functions for safe behavior instead. The error now always uses `WITHOUT_SUGGESTION` (message is just the range description).

## Change

- **error-conditions.json**: Removed the `WITH_SUGGESTION` subclass from `DATETIME_FIELD_OUT_OF_BOUNDS` (kept only `WITHOUT_SUGGESTION`).
- **QueryExecutionErrors.scala**: `ansiDateTimeArgumentOutOfRange` now throws `DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION` with only `rangeMessage` (removed `ansiConfig` parameter).
- **SQL test result files** (date.sql.out, timestamp.sql.out, timestampNTZ/timestamp-ansi.sql.out, postgreSQL/date.sql.out, datetime-legacy.sql.out): Updated expected error class from `WITH_SUGGESTION` to `WITHOUT_SUGGESTION` and removed `ansiConfig` from expected messageParameters where the error is `DATETIME_FIELD_OUT_OF_BOUNDS`.

## Tests

Existing SQL tests for datetime/timestamp/date out-of-bounds errors were updated to expect `DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION` and the new message format (no ansiConfig). No new tests added; behavior change is limited to the error message.

Fixes SPARK-49642

**JIRA assignee for credit:** deepujain

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS [spark]

Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.

MaxGekk commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4853394450

   +1, LGTM. Merging to master/4.x.
   Thank you, @deepujain.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "deepujain (via GitHub)" <gi...@apache.org>.

deepujain commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4558239768

   Rebased on current master. Ran `./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"` and the SQLQueryTestSuite date/timestamp/datetime-legacy/timestamp-ansi filters - passed. Ready for CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "deepujain (via GitHub)" <gi...@apache.org>.

deepujain commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4063206809

   Branch is rebased on master and local SQL/ANSI tests passed. No further changes from my side, ready for review when someone has bandwidth.
   
   ```
   ./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS [spark]

Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.

MaxGekk closed pull request #54695: [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS
URL: https://github.com/apache/spark/pull/54695


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS [spark]

Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.

MaxGekk commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4853409229

   @deepujain Congratulations with your first contribution to Apache Spark!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIELD_OUT_OF_BOUNDS [spark]

Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.

MaxGekk commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4853400874

   **Merge summary** (posted by `merge_spark_pr.py`):
   - merged into master https://github.com/apache/spark/commit/890ce8d6
   - merged into branch-4.x https://github.com/apache/spark/commit/565144b3


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.

MaxGekk commented on code in PR #54695:
URL: https://github.com/apache/spark/pull/54695#discussion_r3388011735


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -305,10 +305,8 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
 
   def ansiDateTimeArgumentOutOfRange(e: Exception): SparkDateTimeException = {
     new SparkDateTimeException(
-      errorClass = "DATETIME_FIELD_OUT_OF_BOUNDS.WITH_SUGGESTION",
-      messageParameters = Map(
-        "rangeMessage" -> e.getMessage,
-        "ansiConfig" -> toSQLConf(SQLConf.ANSI_ENABLED.key)),
+      errorClass = "DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION",

Review Comment:
   After this change, `ansiDateTimeArgumentOutOfRangeWithoutSuggestion` (lines 315–322) has an identical implementation to this method — same `errorClass`, same `messageParameters`, same `context`/`summary`/`cause`. The only remaining difference is the parameter type (`Throwable` vs `Exception`). The `WithoutSuggestion` suffix used to encode a real behavioral distinction; that distinction is gone now that both methods throw `WITHOUT_SUGGESTION`.
   
   Suggested fix (self-contained in this PR):
   1. Widen this method's parameter to `Throwable` (all callers pass `DateTimeException`/`RuntimeException`, compatible on both the Java and Scala sides).
   2. Delete `ansiDateTimeArgumentOutOfRangeWithoutSuggestion` (lines 315–322).
   3. Replace its 3 call sites in `DateTimeUtils.scala` (lines 940, 955, 957) with `ansiDateTimeArgumentOutOfRange`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "deepujain (via GitHub)" <gi...@apache.org>.

deepujain commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4811057371

   Rebased this on current master and updated the PR description to mention the new `QueryExecutionAnsiErrorsSuite` regression test. I reran `./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"` and `./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql"` locally; both passed. Ready for CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "deepujain (via GitHub)" <gi...@apache.org>.

deepujain commented on PR #54695:
URL: https://github.com/apache/spark/pull/54695#issuecomment-4043930595

   Rebased on master. Local run: SQLQueryTestSuite for date/timestamp/datetime-legacy/timestamp-ansi (18 tests) and QueryExecutionAnsiErrorsSuite::SPARK-49642 (1 test)—all passed.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-49642][SQL] Remove the ANSI config suggestion in DATETIME_FIEL… [spark]

Posted by "deepujain (via GitHub)" <gi...@apache.org>.

deepujain commented on code in PR #54695:
URL: https://github.com/apache/spark/pull/54695#discussion_r3409879190


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -305,10 +305,8 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
 
   def ansiDateTimeArgumentOutOfRange(e: Exception): SparkDateTimeException = {
     new SparkDateTimeException(
-      errorClass = "DATETIME_FIELD_OUT_OF_BOUNDS.WITH_SUGGESTION",
-      messageParameters = Map(
-        "rangeMessage" -> e.getMessage,
-        "ansiConfig" -> toSQLConf(SQLConf.ANSI_ENABLED.key)),
+      errorClass = "DATETIME_FIELD_OUT_OF_BOUNDS.WITHOUT_SUGGESTION",

Review Comment:
   Addressed this by widening `ansiDateTimeArgumentOutOfRange` to `Throwable`, deleting `ansiDateTimeArgumentOutOfRangeWithoutSuggestion`, and switching the three `DateTimeUtils` call sites over. Rebased on current master and reran `./build/sbt "sql/testOnly org.apache.spark.sql.errors.QueryExecutionAnsiErrorsSuite -- -z SPARK-49642"` plus `./build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z time.sql"` - both passed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org