[GitHub] [spark] zhenlineo commented on a diff in pull request #40354: [SPARK-42735][CONNECT][SCALA] Allow passing extra confs to RemoteSparkSession

["zhenlineo (via GitHub)" <gi...@apache.org>]

zhenlineo commented on code in PR #40354:
URL: https://github.com/apache/spark/pull/40354#discussion_r1131440189


##########
connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/connect/client/util/RemoteSparkSession.scala:
##########
@@ -114,19 +118,25 @@ object SparkConnectServerUtils {
 }
 
 trait RemoteSparkSession extends ConnectFunSuite with BeforeAndAfterAll {
-  import SparkConnectServerUtils._
   var spark: SparkSession = _
+  var server: SparkConnectServer = _
+
+  protected def sparkConf: SparkConf = new SparkConf(loadDefaults = false)
 
   override def beforeAll(): Unit = {
     super.beforeAll()
-    SparkConnectServerUtils.start()
-    spark = SparkSession.builder().client(SparkConnectClient.builder().port(port).build()).build()
+    server = new SparkConnectServer(sparkConf)

Review Comment:
   This would make each test suite who extend the RemoteSparkSession start a new server. The server start time is pretty bad ~1 min.
   
   Can we have another impl of RemoteSparkSession e.g. `RemoteSparkSessionWithConf` which take extra configs and start new server on demands. But leave the original `RemoteSparkSession` share one server only?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org