You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2020/04/11 15:26:35 UTC
[spark] branch branch-3.0 updated:
[SPARK-31291][SQL][TEST][FOLLOWUP] Fix resource loading error in
ThriftServerQueryTestSuite
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new a077850 [SPARK-31291][SQL][TEST][FOLLOWUP] Fix resource loading error in ThriftServerQueryTestSuite
a077850 is described below
commit a0778501aba8177f65082b2a520875951af21068
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sat Apr 11 08:23:59 2020 -0700
[SPARK-31291][SQL][TEST][FOLLOWUP] Fix resource loading error in ThriftServerQueryTestSuite
### What changes were proposed in this pull request?
[SPARK-31291](https://github.com/apache/spark/pull/28060) broke `ThriftServerQueryTestSuite` in Maven environment. This PR fixes it by copying the resource file from jars to local temp file.
### Why are the changes needed?
To recover the Jenkins jobs in `master` and `branch-3.0`.
- https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-3.0-test-maven-hadoop-2.7-hive-2.3/211/
```
org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite *** ABORTED ***
...
java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute URI: jar:file:/home/jenkins/workspace/spark-branch-3.0-test-maven-hadoop-2.7-hive-2.3/sql/core/target/
spark-sql_2.12-3.0.1-SNAPSHOT-tests.jar!/test-data/postgresql/agg.data
```
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass the Jenkins with SBT and Maven.
- [x] Sbt (`Test build #121117` https://github.com/apache/spark/pull/28186#issuecomment-612329068)
- [x] Maven (`Test build #121118` https://github.com/apache/spark/pull/28186#issuecomment-612414382)
Closes #28186 from dongjoon-hyun/SPARK-31291.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit b4c438a5e09a7b1cd98f9b53e883f339367cfda8)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../sql/hive/thriftserver/ThriftServerQueryTestSuite.scala | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala
index 48e3485..15cc310 100644
--- a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala
+++ b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala
@@ -23,6 +23,7 @@ import java.util.{Locale, MissingFormatArgumentException}
import scala.util.control.NonFatal
+import org.apache.commons.io.FileUtils
import org.apache.commons.lang3.exception.ExceptionUtils
import org.apache.spark.SparkException
@@ -53,6 +54,15 @@ import org.apache.spark.sql.types._
*/
class ThriftServerQueryTestSuite extends SQLQueryTestSuite with SharedThriftServer {
+ override protected def testFile(fileName: String): String = {
+ val url = Thread.currentThread().getContextClassLoader.getResource(fileName)
+ // Copy to avoid URISyntaxException during accessing the resources in `sql/core`
+ val file = File.createTempFile("thriftserver-test", ".data")
+ file.deleteOnExit()
+ FileUtils.copyURLToFile(url, file)
+ file.getAbsolutePath
+ }
+
/** List of test cases to ignore, in lower cases. */
override def blackList: Set[String] = super.blackList ++ Set(
// Missing UDF
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org