You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/09/22 19:39:36 UTC
[GitHub] [spark] MaxGekk opened a new pull request, #37973: [WIP][SPARK-40540][SQL] Migrate compilation errors onto error classes
MaxGekk opened a new pull request, #37973:
URL: https://github.com/apache/spark/pull/37973
### What changes were proposed in this pull request?
In the PR, I propose to migrate all compilation errors onto temporary error classes with the prefix `_LEGACY_ERROR_TEMP_`. The error message will not include the error classes, so, in this way we will preserve the existing behaviour.
### Why are the changes needed?
The migration on temporary error classes allows to gather statistics about errors and detect most popular error classes.
### Does this PR introduce _any_ user-facing change?
No. The error messages should be almost the same by default.
### How was this patch tested?
By running the modified test suites:
```
$ build/sbt "core/testOnly *SparkThrowableSuite"
$ build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk closed pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk closed pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
URL: https://github.com/apache/spark/pull/37973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on code in PR #37973:
URL: https://github.com/apache/spark/pull/37973#discussion_r979056137
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
@amaliujia Also see https://github.com/apache/spark/pull/37916#issuecomment-1251888183
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on PR #37973:
URL: https://github.com/apache/spark/pull/37973#issuecomment-1257593979
Merging to master. Thank you, @cloud-fan for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia commented on a diff in pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
amaliujia commented on code in PR #37973:
URL: https://github.com/apache/spark/pull/37973#discussion_r979037241
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
Maybe it will cause too much work for the migration. If this does not bring much values please feel free to ignore.
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
just wondering why we choose use numbers than a descriptive way for the names?
E.g. `_LEGACY_ERROR_TEMP_1000` => `_LEGACY_ERROR_TEMP_LEGACY_STORE_ASSIGNMENT_POLICY_DISALLOWED`?
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala:
##########
@@ -340,7 +340,7 @@ class AnalysisErrorSuite extends AnalysisTest {
errorTest(
"unresolved star expansion in max",
testRelation2.groupBy($"a")(sum(UnresolvedStar(None))),
- "Invalid usage of '*'" :: "in expression 'sum'" :: Nil)
+ "Invalid usage of '*' in expression 'sum'." :: Nil)
Review Comment:
Nice catch! I did a full text search and only found one place which is here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on code in PR #37973:
URL: https://github.com/apache/spark/pull/37973#discussion_r979051903
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
> The migration on temporary error classes allows to gather statistics about errors and detect most popular error classes.
@amaliujia Please, see in PR's description:
> Why are the changes needed?
The migration on temporary error classes allows to gather statistics about errors and detect most popular error classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia commented on a diff in pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
amaliujia commented on code in PR #37973:
URL: https://github.com/apache/spark/pull/37973#discussion_r979061997
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
Thanks for the context! I see it better now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on a diff in pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on code in PR #37973:
URL: https://github.com/apache/spark/pull/37973#discussion_r979051903
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -1163,5 +1163,257 @@
"message" : [
"<msg>."
]
+ },
+ "_LEGACY_ERROR_TEMP_1000" : {
Review Comment:
> just wondering why we choose use numbers ...
@amaliujia Please, see in PR's description:
> Why are the changes needed?
The migration on temporary error classes allows to gather statistics about errors and detect most popular error classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #37973: [SPARK-40540][SQL] Migrate compilation errors onto error classes
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on PR #37973:
URL: https://github.com/apache/spark/pull/37973#issuecomment-1256250482
@cloud-fan @itholic @srielau @anchovYu @gatorsmile Could you review this PR, please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org