You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/11/21 13:12:04 UTC
[GitHub] [spark] MaxGekk opened a new pull request, #38744: [WIP][SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
MaxGekk opened a new pull request, #38744:
URL: https://github.com/apache/spark/pull/38744
### What changes were proposed in this pull request?
TODO
### Why are the changes needed?
To improve user experience with Spark SQL, in particular, the PR makes errors more clear.
### Does this PR introduce _any_ user-facing change?
Yes, it affects the user-facing errors.
### How was this patch tested?
By running the modified test suites:
```
$ build/sbt "core/testOnly *SparkThrowableSuite"
$ PYSPARK_PYTHON=python3 build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on PR #38744:
URL: https://github.com/apache/spark/pull/38744#issuecomment-1322275421
@panbingkun @LuciferYang @itholic @cloud-fan @srielau Could you review this PR, please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on PR #38744:
URL: https://github.com/apache/spark/pull/38744#issuecomment-1322303670
@panbingkun @LuciferYang Your PRs are related to this one. I would propose to merge this first of all to properly propagate `AnalysisException`s w/ error classes:
- https://github.com/apache/spark/pull/38737
- https://github.com/apache/spark/pull/38725
- https://github.com/apache/spark/pull/38730
- https://github.com/apache/spark/pull/38710
- https://github.com/apache/spark/pull/38707
@cloud-fan We bypass `SparkThrowable` in other places already. In this PR, I propose to do the same instead of additionally wrap them and increase levels of nesting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on PR #38744:
URL: https://github.com/apache/spark/pull/38744#issuecomment-1323205514
Merging to master. Thank you, @panbingkun @LuciferYang @cloud-fan for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] panbingkun commented on pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
panbingkun commented on PR #38744:
URL: https://github.com/apache/spark/pull/38744#issuecomment-1323142938
+1, LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on code in PR #38744:
URL: https://github.com/apache/spark/pull/38744#discussion_r1028220801
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala:
##########
@@ -3393,4 +3393,15 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase {
"unsupported" -> unsupported.toString,
"class" -> unsupported.getClass.toString))
}
+
+ def funcBuildError(funcName: String, cause: Exception): Throwable = {
+ cause.getCause match {
+ case st: SparkThrowable with Throwable => st
Review Comment:
If some Spark's exception happens during preparation of a function call, we just propagate it to users as is otherwise (something we didn't catch) we wrap it by `AnalysisException(errorClass = FAILED_FUNCTION_CALL)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk closed pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk closed pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
URL: https://github.com/apache/spark/pull/38744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on code in PR #38744:
URL: https://github.com/apache/spark/pull/38744#discussion_r1028220801
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala:
##########
@@ -3393,4 +3393,15 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase {
"unsupported" -> unsupported.toString,
"class" -> unsupported.getClass.toString))
}
+
+ def funcBuildError(funcName: String, cause: Exception): Throwable = {
+ cause.getCause match {
+ case st: SparkThrowable with Throwable => st
Review Comment:
If some Spark's exception happens during preparation of a function call, we just propagate it to users AS IS otherwise (something we didn't catch) we wrap it by `AnalysisException(errorClass = FAILED_FUNCTION_CALL)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38744: [SPARK-41217][SQL] Add the error class `FAILED_FUNCTION_CALL`
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38744:
URL: https://github.com/apache/spark/pull/38744#issuecomment-1323206350
OK, I will rebase my pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org