You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/06/25 16:55:58 UTC
[spark] branch master updated: [SPARK-44174][SQL][TESTS] Fix `QueryExecutionErrorsSuite` for Java 21
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b482d693c6d [SPARK-44174][SQL][TESTS] Fix `QueryExecutionErrorsSuite` for Java 21
b482d693c6d is described below
commit b482d693c6de76b583393198fdd043e087ce30f8
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Sun Jun 25 09:55:47 2023 -0700
[SPARK-44174][SQL][TESTS] Fix `QueryExecutionErrorsSuite` for Java 21
### What changes were proposed in this pull request?
This pr change to use new `functionNameRegex` to fix the following test case in `QueryExecutionErrorsSuite` for Java 21:
- `FAILED_EXECUTE_UDF: execute user defined function with registered UDF`
- `FAILED_EXECUTE_UDF: execute user defined function`
### Why are the changes needed?
After https://bugs.openjdk.org/browse/JDK-8292914
The format of the lambda-class name change from
`<declaring_class>$$Lambda$<counter>/#<hidden_class_reference>`
to
`<declaring_class>$$Lambda/#<hidden_class_reference>`
So the check condition of test cases should be changed accordingly for Java 21
### Does this PR introduce _any_ user-facing change?
No, just for test
### How was this patch tested?
- Pass GitHub Actions
- Checked
```
java -version
openjdk version "21-ea" 2023-09-19
OpenJDK Runtime Environment Zulu21+57-CA (build 21-ea+22)
OpenJDK 64-Bit Server VM Zulu21+57-CA (build 21-ea+22, mixed mode, sharing)
```
```
build/sbt clean "sql/testOnly org.apache.spark.sql.errors.QueryExecutionErrorsSuite"
```
Before
```
[info] - FAILED_EXECUTE_UDF: execute user defined function with registered UDF *** FAILED *** (78 milliseconds)
[info] java.lang.IllegalArgumentException: For parameter 'functionName' value '`luckyCharOfWord (QueryExecutionErrorsSuite$$Lambda/0x00000008022254b8)`' does not match: `luckyCharOfWord \(QueryExecutionErrorsSuite\$\$Lambda\$\d+/\w+\)`
[info] at org.apache.spark.SparkFunSuite.$anonfun$checkError$2(SparkFunSuite.scala:333)
[info] at org.apache.spark.SparkFunSuite.$anonfun$checkError$2$adapted(SparkFunSuite.scala:328)
[info] at scala.collection.Iterator.foreach(Iterator.scala:943)
[info] at scala.collection.Iterator.foreach$(Iterator.scala:943)
[info] at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
[info] at scala.collection.IterableLike.foreach(IterableLike.scala:74)
[info] at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
[info] at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
...
[info] - FAILED_EXECUTE_UDF: execute user defined function *** FAILED *** (33 milliseconds)
[info] java.lang.IllegalArgumentException: For parameter 'functionName' value '`QueryExecutionErrorsSuite$$Lambda/0x000000080225d710`' does not match: `QueryExecutionErrorsSuite\$\$Lambda\$\d+/\w+`
[info] at org.apache.spark.SparkFunSuite.$anonfun$checkError$2(SparkFunSuite.scala:333)
[info] at org.apache.spark.SparkFunSuite.$anonfun$checkError$2$adapted(SparkFunSuite.scala:328)
[info] at scala.collection.Iterator.foreach(Iterator.scala:943)
[info] at scala.collection.Iterator.foreach$(Iterator.scala:943)
[info] at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
...
[info] Run completed in 9 seconds, 467 milliseconds.
[info] Total number of tests run: 39
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 37, failed 2, canceled 0, ignored 0, pending 0
[info] *** 2 TESTS FAILED ***
[error] Failed tests:
[error] org.apache.spark.sql.errors.QueryExecutionErrorsSuite
[error] (sql / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
```
After
```
[info] Run completed in 9 seconds, 678 milliseconds.
[info] Total number of tests run: 39
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 39, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
```
Closes #41722 from LuciferYang/QueryExecutionErrorsSuite-Java21.
Authored-by: yangjie01 <ya...@baidu.com>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../spark/sql/errors/QueryExecutionErrorsSuite.scala | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
index 8f47b06d855..37cdfcf10c7 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
@@ -409,13 +409,17 @@ class QueryExecutionErrorsSuite
spark.sql("select luckyCharOfWord(word, index) from words").collect()
}
assert(e.getCause.isInstanceOf[SparkException])
+ val functionNameRegex = if (Utils.isJavaVersionAtLeast21) {
+ "`luckyCharOfWord \\(QueryExecutionErrorsSuite\\$\\$Lambda/\\w+\\)`"
+ } else {
+ "`luckyCharOfWord \\(QueryExecutionErrorsSuite\\$\\$Lambda\\$\\d+/\\w+\\)`"
+ }
checkError(
exception = e.getCause.asInstanceOf[SparkException],
errorClass = "FAILED_EXECUTE_UDF",
parameters = Map(
- "functionName" ->
- "`luckyCharOfWord \\(QueryExecutionErrorsSuite\\$\\$Lambda\\$\\d+/\\w+\\)`",
+ "functionName" -> functionNameRegex,
"signature" -> "string, int",
"result" -> "string"),
matchPVals = true)
@@ -430,11 +434,16 @@ class QueryExecutionErrorsSuite
words.select(luckyCharOfWord($"word", $"index")).collect()
}
assert(e.getCause.isInstanceOf[SparkException])
+ val functionNameRegex = if (Utils.isJavaVersionAtLeast21) {
+ "`QueryExecutionErrorsSuite\\$\\$Lambda/\\w+`"
+ } else {
+ "`QueryExecutionErrorsSuite\\$\\$Lambda\\$\\d+/\\w+`"
+ }
checkError(
exception = e.getCause.asInstanceOf[SparkException],
errorClass = "FAILED_EXECUTE_UDF",
- parameters = Map("functionName" -> "`QueryExecutionErrorsSuite\\$\\$Lambda\\$\\d+/\\w+`",
+ parameters = Map("functionName" -> functionNameRegex,
"signature" -> "string, int",
"result" -> "string"),
matchPVals = true)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org