You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2023/01/02 14:19:49 UTC
[GitHub] [spark] MaxGekk commented on a diff in pull request #39284: [SPARK-41573][SQL] Assign name to _LEGACY_ERROR_TEMP_2136
MaxGekk commented on code in PR #39284:
URL: https://github.com/apache/spark/pull/39284#discussion_r1060057744
##########
sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala:
##########
@@ -774,15 +775,21 @@ class JsonFunctionsSuite extends QueryTest with SharedSparkSession {
df.select(from_json($"value", schema, Map("mode" -> "PERMISSIVE"))),
Row(Row(null, 11, badRec)) :: Row(Row(2, 12, null)) :: Nil)
- val errMsg = intercept[SparkException] {
+ val exception = intercept[SparkException] {
df.select(from_json($"value", schema, Map("mode" -> "FAILFAST"))).collect()
- }.getMessage
+ }
- assert(errMsg.contains(
+ assert(exception.getMessage.contains(
"Malformed records are detected in record parsing. Parse Mode: FAILFAST."))
- assert(errMsg.contains(
- "Failed to parse field name a, field value 1, " +
- "[VALUE_STRING] to target spark data type [IntegerType]."))
+ checkError(
+ exception = ExceptionUtils.getRootCause(exception).asInstanceOf[SparkRuntimeException],
+ errorClass = "CANNOT_PARSE_JSON_FIELD",
+ parameters = Map(
+ "fieldName" -> "a",
+ "fieldValue" -> "1",
+ "token" -> "VALUE_STRING",
+ "dataType" -> "IntegerType")
Review Comment:
Could you use `toSQLType()`, please.
##########
core/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,11 @@
],
"sqlState" : "42000"
},
+ "CANNOT_PARSE_JSON_FIELD" : {
+ "message" : [
+ "Failed to parse field name <fieldName>, field value <fieldValue>, token <token> to target spark data type <dataType>."
Review Comment:
```suggestion
"Cannot parse the field name <fieldName> and the value <fieldValue> of the JSON token type <jsonType> to target Spark data type <dataType>."
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org