You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "micheal-o (via GitHub)" <gi...@apache.org> on 2024/02/28 01:53:25 UTC
[PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
micheal-o opened a new pull request, #45299:
URL: https://github.com/apache/spark/pull/45299
### What changes were proposed in this pull request?
Any exception can be thrown from the user provided function for ForEachBatchSink. We want to classify this class of errors. Including errors from Python (Py4JException) and Scala functions.
### Why are the changes needed?
The user provided function can throw any type of error. Using the new error framework for better error messages and classification.
### Does this PR introduce _any_ user-facing change?
Yes, better error message with error class for ForeachBatchSink user function failures.
### How was this patch tested?
Updated existing tests and added a new one. Covers python and Scala.
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
Posted by "micheal-o (via GitHub)" <gi...@apache.org>.
micheal-o commented on code in PR #45299:
URL: https://github.com/apache/spark/pull/45299#discussion_r1506754573
##########
docs/sql-error-conditions.md:
##########
@@ -1148,7 +1154,7 @@ Please increase executor memory using the --executor-memory option or "`<config>
[SQLSTATE: 42001](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)
-Found an invalid expression encoder. Expects an instance of `ExpressionEncoder` but got `<encoderType>`. For more information consult '`<docroot>`/api/java/index.html?org/apache/spark/sql/Encoder.html'.
+Found an invalid expression encoder. Expects an instance of ExpressionEncoder but got `<encoderType>`. For more information consult '`<docroot>`/api/java/index.html?org/apache/spark/sql/Encoder.html'.
Review Comment:
@MaxGekk done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
Posted by "micheal-o (via GitHub)" <gi...@apache.org>.
micheal-o commented on PR #45299:
URL: https://github.com/apache/spark/pull/45299#issuecomment-1970211086
cc @HeartSaVioR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR closed pull request #45299: [SPARK-47200][SS] Error class for Foreach batch sink user function error
URL: https://github.com/apache/spark/pull/45299
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on code in PR #45299:
URL: https://github.com/apache/spark/pull/45299#discussion_r1506474099
##########
docs/sql-error-conditions.md:
##########
@@ -1148,7 +1154,7 @@ Please increase executor memory using the --executor-memory option or "`<config>
[SQLSTATE: 42001](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)
-Found an invalid expression encoder. Expects an instance of `ExpressionEncoder` but got `<encoderType>`. For more information consult '`<docroot>`/api/java/index.html?org/apache/spark/sql/Encoder.html'.
+Found an invalid expression encoder. Expects an instance of ExpressionEncoder but got `<encoderType>`. For more information consult '`<docroot>`/api/java/index.html?org/apache/spark/sql/Encoder.html'.
Review Comment:
Similar changes made by https://github.com/apache/spark/pull/45226. Could you rebase your PR on the recent master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47200][SS] Error class for Foreach batch sink user function error [spark]
Posted by "HeartSaVioR (via GitHub)" <gi...@apache.org>.
HeartSaVioR commented on PR #45299:
URL: https://github.com/apache/spark/pull/45299#issuecomment-1972088992
Thanks! Merging to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org