You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "yuexing (via GitHub)" <gi...@apache.org> on 2025/06/08 06:35:31 UTC

[PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

yuexing opened a new pull request, #51113:
URL: https://github.com/apache/spark/pull/51113

   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
     2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
     3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
     4. Be sure to keep the PR description updated to reflect all changes.
     5. Please write your PR title to summarize what this PR proposes.
     6. If possible, provide a concise example to reproduce the issue for a faster review.
     7. If you want to add a new configuration, please read the guideline first for naming configurations in
        'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
     8. If you want to add or modify an error type or message, please read the guideline first in
        'common/utils/src/main/resources/error/README.md'.
   -->
   
   ### What changes were proposed in this pull request?
   see https://issues.apache.org/jira/browse/SPARK-42329. Assign a meaningful name to error _LEGACY_ERROR_TEMP_2256.
   
   ### Why are the changes needed?
   see https://issues.apache.org/jira/browse/SPARK-42329
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   UT, there's an existing UT.
   
   
   ### Was this patch authored or co-authored using generative AI tooling?
   No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2152393220


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -2152,11 +2152,20 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
       cause = failureReason)
   }
 
-  def foreachWriterAbortedDueToTaskFailureError(): Throwable = {
+  def foreachWriterAbortedDueToTaskFailureError(errors: Seq[Throwable] = Seq.empty): Throwable = {

Review Comment:
   oh, wait! Let me revert the change. I shouldn't have changed this interface. 
   
   foreachWriterAbortedDueToTaskFailureError is **only** called when an error happened **but not** during write stage, as in code
   
   ```
   class ForeachDataWriter[T] {
     ...
     def abort(): Unit = {
       if (errorOrNull == null) {
         errorOrNull = QueryExecutionErrors.foreachWriterAbortedDueToTaskFailureError()
       }
     }
   
   ```
   
   when the error happened in write, it will be captured in errorOrNull, and is handled by close
   ```
   class ForeachDataWriter[T] {
     ...
     override def close(): Unit = {
       writer.close(errorOrNull)
     }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #51113: [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256
URL: https://github.com/apache/spark/pull/51113


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "mihailom-db (via GitHub)" <gi...@apache.org>.
mihailom-db commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2134714046


##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1775,6 +1775,12 @@
     ],
     "sqlState" : "39000"
   },
+  "FOREACH_WRITER_ABORTED" : {
+    "message" : [
+      "Foreach writer has been aborted due to a task failure."

Review Comment:
   Could we take a look at adding a hint for the user on how to actually overcome this issue? Is it possible to point to which task failed or what could prevent it to fail?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2152393220


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -2152,11 +2152,20 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
       cause = failureReason)
   }
 
-  def foreachWriterAbortedDueToTaskFailureError(): Throwable = {
+  def foreachWriterAbortedDueToTaskFailureError(errors: Seq[Throwable] = Seq.empty): Throwable = {

Review Comment:
   oh, wait! Let me revert the change. I shouldn't have changed this interface. 
   
   foreachWriterAbortedDueToTaskFailureError is **only** called when an error happens **but not** during write stage, as in code
   
   ```
   class ForeachDataWriter[T] {
     ...
     def abort(): Unit = {
       if (errorOrNull == null) {
         errorOrNull = QueryExecutionErrors.foreachWriterAbortedDueToTaskFailureError()
       }
     }
   
   ```
   
   when the error happens in write, it will be captured in errorOrNull, and is handled by close
   ```
   class ForeachDataWriter[T] {
     ...
     override def close(): Unit = {
       writer.close(errorOrNull)
     }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2134737597


##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1775,6 +1775,12 @@
     ],
     "sqlState" : "39000"
   },
+  "FOREACH_WRITER_ABORTED" : {
+    "message" : [
+      "Foreach writer has been aborted due to a task failure."

Review Comment:
   hi @mihailom-db , sure. I actually thought about it while renaming, but at that point I was not sure whether it's a good idea to change the msg (exposed to the user) in one commit. Since you also mentioned, I'll just do it ;)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "mihailom-db (via GitHub)" <gi...@apache.org>.
mihailom-db commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2152027324


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -2152,11 +2152,20 @@ private[sql] object QueryExecutionErrors extends QueryErrorsBase with ExecutionE
       cause = failureReason)
   }
 
-  def foreachWriterAbortedDueToTaskFailureError(): Throwable = {
+  def foreachWriterAbortedDueToTaskFailureError(errors: Seq[Throwable] = Seq.empty): Throwable = {

Review Comment:
   We need to update all points where this was called as well. Looking at the code, this might not be a good idea. We already log info on the task that failed and which stage it was in. @cloud-fan what is your take on this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #51113:
URL: https://github.com/apache/spark/pull/51113#issuecomment-3336345420

   We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2152408532


##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1775,6 +1775,12 @@
     ],
     "sqlState" : "39000"
   },
+  "FOREACH_WRITER_ABORTED" : {
+    "message" : [
+      "Foreach writer has been aborted due to a task failure."

Review Comment:
   I just reverted the change, because this error is **only** thrown when there's an error in the earlier stage. 
   In this case, we cannot really capture the cause, and thus cannot show it, right? or do you have a way how to capture the earlier cause? @mihailom-db 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2152408532


##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1775,6 +1775,12 @@
     ],
     "sqlState" : "39000"
   },
+  "FOREACH_WRITER_ABORTED" : {
+    "message" : [
+      "Foreach writer has been aborted due to a task failure."

Review Comment:
   I just reverted the change, because this error is **only** thrown when there's an error in the earlier stage. 
   In this case, we cannot really capture the cause, and thus cannot show it, right? @mihailom-db 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on code in PR #51113:
URL: https://github.com/apache/spark/pull/51113#discussion_r2135305242


##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -1775,6 +1775,12 @@
     ],
     "sqlState" : "39000"
   },
+  "FOREACH_WRITER_ABORTED" : {
+    "message" : [
+      "Foreach writer has been aborted due to a task failure."

Review Comment:
   hi @mihailom-db I did an update, please take a look. Unfortunately, it can only capture error detail during Write now. if the exception is thrown from earlier stage, but abort during Write, then the error detail can't be captured. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-42329][SQL]Assign name to _LEGACY_ERROR_TEMP_2256 [spark]

Posted by "yuexing (via GitHub)" <gi...@apache.org>.
yuexing commented on PR #51113:
URL: https://github.com/apache/spark/pull/51113#issuecomment-2978800542

   friendly ping @mihailom-db 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org