You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2023/11/21 18:27:21 UTC
(spark) branch branch-3.3 updated: [SPARK-46019][SQL][TESTS] Fix `HiveThriftServer2ListenerSuite` and `ThriftServerPageSuite` to create `java.io.tmpdir` if it doesn't exist
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 84e82beefb2 [SPARK-46019][SQL][TESTS] Fix `HiveThriftServer2ListenerSuite` and `ThriftServerPageSuite` to create `java.io.tmpdir` if it doesn't exist
84e82beefb2 is described below
commit 84e82beefb28e9b3a06ff73ca5ecc92ac8c50f3c
Author: panbingkun <pb...@gmail.com>
AuthorDate: Tue Nov 21 10:26:36 2023 -0800
[SPARK-46019][SQL][TESTS] Fix `HiveThriftServer2ListenerSuite` and `ThriftServerPageSuite` to create `java.io.tmpdir` if it doesn't exist
### What changes were proposed in this pull request?
The pr aims to fix `HiveThriftServer2ListenerSuite` and `ThriftServerPageSuite` failed when there are running on local.
```
[info] ThriftServerPageSuite:
[info] - thriftserver page should load successfully *** FAILED *** (515 milliseconds)
[info] java.lang.IllegalStateException: Could not initialize plugin: interface org.mockito.plugins.MockMaker (alternate: null)
[info] at org.mockito.internal.configuration.plugins.PluginLoader$1.invoke(PluginLoader.java:84)
[info] at jdk.proxy2/jdk.proxy2.$Proxy20.isTypeMockable(Unknown Source)
[info] at org.mockito.internal.util.MockUtil.typeMockabilityOf(MockUtil.java:78)
[info] at org.mockito.internal.util.MockCreationValidator.validateType(MockCreationValidator.java:22)
[info] at org.mockito.internal.creation.MockSettingsImpl.validatedSettings(MockSettingsImpl.java:267)
[info] at org.mockito.internal.creation.MockSettingsImpl.build(MockSettingsImpl.java:234)
[info] at org.mockito.internal.MockitoCore.mock(MockitoCore.java:86)
[info] at org.mockito.Mockito.mock(Mockito.java:2037)
[info] at org.mockito.Mockito.mock(Mockito.java:2010)
[info] at org.apache.spark.sql.hive.thriftserver.ui.ThriftServerPageSuite.getStatusStore(ThriftServerPageSuite.scala:49)
```
It can be simply reproduced by running the following command:
```
build/sbt "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.ui.HiveThriftServer2ListenerSuite" -Phive-thriftserver
build/sbt "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.ui.ThriftServerPageSuite" -Phive-thriftserver
```
### Why are the changes needed?
Fix tests failed.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually test:
```
build/sbt "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.ui.HiveThriftServer2ListenerSuite" -Phive-thriftserver
build/sbt "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.ui.ThriftServerPageSuite" -Phive-thriftserver
```
After it:
```
[info] - listener events should store successfully (live = true) (1 second, 711 milliseconds)
[info] - listener events should store successfully (live = false) (6 milliseconds)
[info] - cleanup session if exceeds the threshold (live = true) (21 milliseconds)
[info] - cleanup session if exceeds the threshold (live = false) (3 milliseconds)
[info] - update execution info when jobstart event come after execution end event (9 milliseconds)
[info] - SPARK-31387 - listener update methods should not throw exception with unknown input (8 milliseconds)
[info] Run completed in 3 seconds, 734 milliseconds.
[info] Total number of tests run: 6
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 6, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 156 s (02:36), completed Nov 21, 2023, 1:57:21 PM
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #43921 from panbingkun/SPARK-46019.
Authored-by: panbingkun <pb...@gmail.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
(cherry picked from commit fdcd20f4b51c3ddddaae12f7d3f429e7b77c9f5e)
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala | 10 ++++++++++
.../spark/sql/hive/thriftserver/ui/ThriftServerPageSuite.scala | 10 ++++++++++
2 files changed, 20 insertions(+)
diff --git a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala
index f5167a4ea73..62d97772bcb 100644
--- a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala
+++ b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/HiveThriftServer2ListenerSuite.scala
@@ -17,6 +17,7 @@
package org.apache.spark.sql.hive.thriftserver.ui
+import java.io.File
import java.util.Properties
import org.mockito.Mockito.{mock, RETURNS_SMART_NULLS}
@@ -34,6 +35,15 @@ class HiveThriftServer2ListenerSuite extends SparkFunSuite with BeforeAndAfter {
private var kvstore: ElementTrackingStore = _
+ protected override def beforeAll(): Unit = {
+ val tmpDirName = System.getProperty("java.io.tmpdir")
+ val tmpDir = new File(tmpDirName)
+ if (!tmpDir.exists()) {
+ tmpDir.mkdirs()
+ }
+ super.beforeAll()
+ }
+
after {
if (kvstore != null) {
kvstore.close()
diff --git a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPageSuite.scala b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPageSuite.scala
index d7e18521996..1245e6740eb 100644
--- a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPageSuite.scala
+++ b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/ui/ThriftServerPageSuite.scala
@@ -17,6 +17,7 @@
package org.apache.spark.sql.hive.thriftserver.ui
+import java.io.File
import java.util.{Calendar, Locale}
import javax.servlet.http.HttpServletRequest
@@ -34,6 +35,15 @@ class ThriftServerPageSuite extends SparkFunSuite with BeforeAndAfter {
private var kvstore: ElementTrackingStore = _
+ protected override def beforeAll(): Unit = {
+ val tmpDirName = System.getProperty("java.io.tmpdir")
+ val tmpDir = new File(tmpDirName)
+ if (!tmpDir.exists()) {
+ tmpDir.mkdirs()
+ }
+ super.beforeAll()
+ }
+
after {
if (kvstore != null) {
kvstore.close()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org