You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Max Gekk (Jira)" <ji...@apache.org> on 2022/04/04 09:08:00 UTC
[jira] [Updated] (SPARK-33539) Standardize exception messages in Spark
[ https://issues.apache.org/jira/browse/SPARK-33539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Max Gekk updated SPARK-33539:
-----------------------------
Epic Link: SPARK-38781
> Standardize exception messages in Spark
> ---------------------------------------
>
> Key: SPARK-33539
> URL: https://issues.apache.org/jira/browse/SPARK-33539
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 3.1.0
> Reporter: Allison Wang
> Priority: Major
>
> In the SPIP: Standardize Exception Messages in Spark, there are three major improvements proposed:
> # Group error messages in dedicated files.
> # Establish an error message guideline for developers.
> # Improve error message quality.
> The first step is to centralize error messages for each component into its own dedicated file(s). This can help with auditing error messages and subsequent tasks to establish a guideline and improve message quality in the future.
> A general rule of thumb for grouping exceptions:
> * AnalysisException => QueryCompilationErrors
> * SparkException, RuntimeException(UnsupportedOperationException, IllegalArgumentException...) => QueryExecutionErrors
> * ParseException => QueryParsingErrors
> (A special case for {{Command}}: since commands are executed eagerly, users can immediately see the errors even if the exceptions are thrown in {{SparkPlan.execute}}. In this case, we should group the exceptions in commands as QueryCompilationErrors.)
> Here is an example RP to group all `AnalysisExcpetion` in Analyzer into QueryCompilationErrors: [SPARK-32670|https://github.com/apache/spark/pull/29497]
> Please see the [SPIP|https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing] for more details.
> Exceptions per component:
> ||component||exception||
> |sql|714|
> |core|334|
> |mllib|161|
> |streaming|43|
> |resource-managers|42|
> |external|35|
> |examples|20|
> |mllib-local|10|
> |graphx|6|
> |repl|5|
> |hadoop-cloud|1|
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org