You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/08/03 03:07:24 UTC
[spark] branch branch-3.5 updated: [SPARK-44059][SQL] Add better error messages for SQL named argumnts
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new 0ff363f4323 [SPARK-44059][SQL] Add better error messages for SQL named argumnts
0ff363f4323 is described below
commit 0ff363f43238c8619bb5c0c8a4a75d2e17b699ee
Author: Richard Yu <ri...@databricks.com>
AuthorDate: Thu Aug 3 12:07:05 2023 +0900
[SPARK-44059][SQL] Add better error messages for SQL named argumnts
### What changes were proposed in this pull request?
Correct error messages.
### Why are the changes needed?
Need to have better quality messages.
### Does this PR introduce _any_ user-facing change?
Error messages are more specific.
### How was this patch tested?
Tested in SQLQueryTestSuite: named-function-arguments.sql and NamedParameterFunctionSuite.
Authored-by: Richard Yu <richard.yudatabricks.com>
(cherry picked from commit 228b5dbfd7688a8efa7135d9ec7b00b71e41a38a)
Closes #42177 from learningchess2003/error-messages.
Lead-authored-by: Richard Yu <ri...@databricks.com>
Co-authored-by: Richard Yu <13...@users.noreply.github.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit b390c9c55a0c16423657ba64f27b9a16ce509c23)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.../src/main/resources/error/error-classes.json | 8 ++++----
...ate-routine-parameter-assignment-error-class.md | 4 ++--
docs/sql-error-conditions.md | 4 ++--
.../plans/logical/FunctionBuilderBase.scala | 24 ++++++++++++----------
.../spark/sql/errors/QueryCompilationErrors.scala | 13 ++++++++----
.../analysis/NamedParameterFunctionSuite.scala | 11 ++++++----
.../named-function-arguments.sql.out | 4 +++-
.../results/named-function-arguments.sql.out | 4 +++-
8 files changed, 43 insertions(+), 29 deletions(-)
diff --git a/common/utils/src/main/resources/error/error-classes.json b/common/utils/src/main/resources/error/error-classes.json
index 480ec636283..df425d7b2df 100644
--- a/common/utils/src/main/resources/error/error-classes.json
+++ b/common/utils/src/main/resources/error/error-classes.json
@@ -750,12 +750,12 @@
"subClass" : {
"BOTH_POSITIONAL_AND_NAMED" : {
"message" : [
- "A positional argument and named argument both referred to the same parameter."
+ "A positional argument and named argument both referred to the same parameter. Please remove the named argument referring to this parameter."
]
},
"DOUBLE_NAMED_ARGUMENT_REFERENCE" : {
"message" : [
- "More than one named argument referred to the same parameter."
+ "More than one named argument referred to the same parameter. Please assign a value only once."
]
}
},
@@ -2387,7 +2387,7 @@
},
"REQUIRED_PARAMETER_NOT_FOUND" : {
"message" : [
- "Cannot invoke function <functionName> because the parameter named <parameterName> is required, but the function call did not supply a value. Please update the function call to supply an argument value (either positionally or by name) and retry the query again."
+ "Cannot invoke function <functionName> because the parameter named <parameterName> is required, but the function call did not supply a value. Please update the function call to supply an argument value (either positionally at index <index> or by name) and retry the query again."
],
"sqlState" : "4274K"
},
@@ -2583,7 +2583,7 @@
},
"UNEXPECTED_POSITIONAL_ARGUMENT" : {
"message" : [
- "Cannot invoke function <functionName> because it contains positional argument(s) following named argument(s); please rearrange them so the positional arguments come first and then retry the query again."
+ "Cannot invoke function <functionName> because it contains positional argument(s) following the named argument assigned to <parameterName>; please rearrange them so the positional arguments come first and then retry the query again."
],
"sqlState" : "4274K"
},
diff --git a/docs/sql-error-conditions-duplicate-routine-parameter-assignment-error-class.md b/docs/sql-error-conditions-duplicate-routine-parameter-assignment-error-class.md
index d9f14b5a55e..eb5ca2a0169 100644
--- a/docs/sql-error-conditions-duplicate-routine-parameter-assignment-error-class.md
+++ b/docs/sql-error-conditions-duplicate-routine-parameter-assignment-error-class.md
@@ -27,10 +27,10 @@ This error class has the following derived error classes:
## BOTH_POSITIONAL_AND_NAMED
-A positional argument and named argument both referred to the same parameter.
+A positional argument and named argument both referred to the same parameter. Please remove the named argument referring to this parameter.
## DOUBLE_NAMED_ARGUMENT_REFERENCE
-More than one named argument referred to the same parameter.
+More than one named argument referred to the same parameter. Please assign a value only once.
diff --git a/docs/sql-error-conditions.md b/docs/sql-error-conditions.md
index 1edab4797ab..9e2a484d057 100644
--- a/docs/sql-error-conditions.md
+++ b/docs/sql-error-conditions.md
@@ -1555,7 +1555,7 @@ The `<clause>` clause may be used at most once per `<operation>` operation.
[SQLSTATE: 4274K](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)
-Cannot invoke function `<functionName>` because the parameter named `<parameterName>` is required, but the function call did not supply a value. Please update the function call to supply an argument value (either positionally or by name) and retry the query again.
+Cannot invoke function `<functionName>` because the parameter named `<parameterName>` is required, but the function call did not supply a value. Please update the function call to supply an argument value (either positionally at index `<index>` or by name) and retry the query again.
### REQUIRES_SINGLE_PART_NAMESPACE
@@ -1764,7 +1764,7 @@ Parameter `<paramIndex>` of function `<functionName>` requires the `<requiredTyp
[SQLSTATE: 4274K](sql-error-conditions-sqlstates.html#class-42-syntax-error-or-access-rule-violation)
-Cannot invoke function `<functionName>` because it contains positional argument(s) following named argument(s); please rearrange them so the positional arguments come first and then retry the query again.
+Cannot invoke function `<functionName>` because it contains positional argument(s) following the named argument assigned to `<parameterName>`; please rearrange them so the positional arguments come first and then retry the query again.
### UNKNOWN_PROTOBUF_MESSAGE_TYPE
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/FunctionBuilderBase.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/FunctionBuilderBase.scala
index 4a2b9eae981..1088655f60c 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/FunctionBuilderBase.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/FunctionBuilderBase.scala
@@ -104,7 +104,7 @@ object NamedParametersSupport {
val positionalParametersSet = allParameterNames.take(positionalArgs.size).toSet
val namedParametersSet = collection.mutable.Set[String]()
- for (arg <- namedArgs) {
+ namedArgs.zipWithIndex.foreach { case (arg, index) =>
arg match {
case namedArg: NamedArgumentExpression =>
val parameterName = namedArg.key
@@ -122,7 +122,8 @@ object NamedParametersSupport {
}
namedParametersSet.add(namedArg.key)
case _ =>
- throw QueryCompilationErrors.unexpectedPositionalArgument(functionName)
+ throw QueryCompilationErrors.unexpectedPositionalArgument(
+ functionName, namedArgs(index - 1).asInstanceOf[NamedArgumentExpression].key)
}
}
@@ -141,15 +142,16 @@ object NamedParametersSupport {
}.toMap
// We rearrange named arguments to match their positional order.
- val rearrangedNamedArgs: Seq[Expression] = namedParameters.map { param =>
- namedArgMap.getOrElse(
- param.name,
- if (param.default.isEmpty) {
- throw QueryCompilationErrors.requiredParameterNotFound(functionName, param.name)
- } else {
- param.default.get
- }
- )
+ val rearrangedNamedArgs: Seq[Expression] = namedParameters.zipWithIndex.map {
+ case (param, index) =>
+ namedArgMap.getOrElse(
+ param.name,
+ if (param.default.isEmpty) {
+ throw QueryCompilationErrors.requiredParameterNotFound(functionName, param.name, index)
+ } else {
+ param.default.get
+ }
+ )
}
val rearrangedArgs = positionalArgs ++ rearrangedNamedArgs
assert(rearrangedArgs.size == parameters.size)
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
index 010b4ec92a9..91548917b2f 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
@@ -90,12 +90,13 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase with Compilat
}
def requiredParameterNotFound(
- functionName: String, parameterName: String) : Throwable = {
+ functionName: String, parameterName: String, index: Int) : Throwable = {
new AnalysisException(
errorClass = "REQUIRED_PARAMETER_NOT_FOUND",
messageParameters = Map(
"functionName" -> toSQLId(functionName),
- "parameterName" -> toSQLId(parameterName))
+ "parameterName" -> toSQLId(parameterName),
+ "index" -> index.toString)
)
}
@@ -115,10 +116,14 @@ private[sql] object QueryCompilationErrors extends QueryErrorsBase with Compilat
)
}
- def unexpectedPositionalArgument(functionName: String): Throwable = {
+ def unexpectedPositionalArgument(
+ functionName: String,
+ precedingNamedArgument: String): Throwable = {
new AnalysisException(
errorClass = "UNEXPECTED_POSITIONAL_ARGUMENT",
- messageParameters = Map("functionName" -> toSQLId(functionName))
+ messageParameters = Map(
+ "functionName" -> toSQLId(functionName),
+ "parameterName" -> toSQLId(precedingNamedArgument))
)
}
diff --git a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/NamedParameterFunctionSuite.scala b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/NamedParameterFunctionSuite.scala
index dd5cb5e7d03..99fed4d2ee5 100644
--- a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/NamedParameterFunctionSuite.scala
+++ b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/NamedParameterFunctionSuite.scala
@@ -40,6 +40,7 @@ case class DummyExpression(
}
object DummyExpressionBuilder extends ExpressionBuilder {
+
def defaultFunctionSignature: FunctionSignature = {
FunctionSignature(Seq(InputParameter("k1"),
InputParameter("k2"),
@@ -49,11 +50,12 @@ object DummyExpressionBuilder extends ExpressionBuilder {
override def functionSignature: Option[FunctionSignature] =
Some(defaultFunctionSignature)
+
override def build(funcName: String, expressions: Seq[Expression]): Expression =
DummyExpression(expressions(0), expressions(1), expressions(2), expressions(3))
}
-class NamedArgumentFunctionSuite extends AnalysisTest {
+class NamedParameterFunctionSuite extends AnalysisTest {
final val k1Arg = Literal("v1")
final val k2Arg = NamedArgumentExpression("k2", Literal("v2"))
@@ -61,6 +63,7 @@ class NamedArgumentFunctionSuite extends AnalysisTest {
final val k4Arg = NamedArgumentExpression("k4", Literal("v4"))
final val namedK1Arg = NamedArgumentExpression("k1", Literal("v1-2"))
final val args = Seq(k1Arg, k4Arg, k2Arg, k3Arg)
+
final val expectedSeq = Seq(Literal("v1"), Literal("v2"), Literal("v3"), Literal("v4"))
final val signature = DummyExpressionBuilder.defaultFunctionSignature
final val illegalSignature = FunctionSignature(Seq(
@@ -115,8 +118,8 @@ class NamedArgumentFunctionSuite extends AnalysisTest {
checkError(
exception = parseRearrangeException(signature, Seq(k1Arg, k2Arg, k3Arg), "foo"),
errorClass = "REQUIRED_PARAMETER_NOT_FOUND",
- parameters = Map("functionName" -> toSQLId("foo"), "parameterName" -> toSQLId("k4"))
- )
+ parameters = Map(
+ "functionName" -> toSQLId("foo"), "parameterName" -> toSQLId("k4"), "index" -> "2"))
}
test("UNRECOGNIZED_PARAMETER_NAME") {
@@ -134,7 +137,7 @@ class NamedArgumentFunctionSuite extends AnalysisTest {
exception = parseRearrangeException(signature,
Seq(k2Arg, k3Arg, k1Arg, k4Arg), "foo"),
errorClass = "UNEXPECTED_POSITIONAL_ARGUMENT",
- parameters = Map("functionName" -> toSQLId("foo"))
+ parameters = Map("functionName" -> toSQLId("foo"), "parameterName" -> toSQLId("k3"))
)
}
diff --git a/sql/core/src/test/resources/sql-tests/analyzer-results/named-function-arguments.sql.out b/sql/core/src/test/resources/sql-tests/analyzer-results/named-function-arguments.sql.out
index 650b61b4192..11e2651c6f2 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/named-function-arguments.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/named-function-arguments.sql.out
@@ -229,7 +229,8 @@ org.apache.spark.sql.AnalysisException
"errorClass" : "UNEXPECTED_POSITIONAL_ARGUMENT",
"sqlState" : "4274K",
"messageParameters" : {
- "functionName" : "`mask`"
+ "functionName" : "`mask`",
+ "parameterName" : "`lowerChar`"
},
"queryContext" : [ {
"objectType" : "",
@@ -292,6 +293,7 @@ org.apache.spark.sql.AnalysisException
"sqlState" : "4274K",
"messageParameters" : {
"functionName" : "`mask`",
+ "index" : "0",
"parameterName" : "`str`"
},
"queryContext" : [ {
diff --git a/sql/core/src/test/resources/sql-tests/results/named-function-arguments.sql.out b/sql/core/src/test/resources/sql-tests/results/named-function-arguments.sql.out
index 77c15b56c8d..60301862a35 100644
--- a/sql/core/src/test/resources/sql-tests/results/named-function-arguments.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/named-function-arguments.sql.out
@@ -214,7 +214,8 @@ org.apache.spark.sql.AnalysisException
"errorClass" : "UNEXPECTED_POSITIONAL_ARGUMENT",
"sqlState" : "4274K",
"messageParameters" : {
- "functionName" : "`mask`"
+ "functionName" : "`mask`",
+ "parameterName" : "`lowerChar`"
},
"queryContext" : [ {
"objectType" : "",
@@ -283,6 +284,7 @@ org.apache.spark.sql.AnalysisException
"sqlState" : "4274K",
"messageParameters" : {
"functionName" : "`mask`",
+ "index" : "0",
"parameterName" : "`str`"
},
"queryContext" : [ {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org