You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2023/03/24 01:56:52 UTC
[spark] branch branch-3.4 updated: [SPARK-42202][CONNECT][TEST][FOLLOWUP] Loop around command entry in SimpleSparkConnectService
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new f20a269695a [SPARK-42202][CONNECT][TEST][FOLLOWUP] Loop around command entry in SimpleSparkConnectService
f20a269695a is described below
commit f20a269695aef73779cfbaa83d05575e0bbde3af
Author: Juliusz Sompolski <ju...@databricks.com>
AuthorDate: Fri Mar 24 10:56:26 2023 +0900
[SPARK-42202][CONNECT][TEST][FOLLOWUP] Loop around command entry in SimpleSparkConnectService
### What changes were proposed in this pull request?
With the while loop around service startup, any ENTER hit in the SimpleSparkConnectService console made it loop around, try to start the service anew, and fail with address already in use.
Change the loop to be around the `StdIn.readline()` entry.
### Why are the changes needed?
Better testing / development / debugging with SparkConnect
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual use of `connector/connect/bin/spark-connect`
Closes #40537 from juliuszsompolski/SPARK-42202-followup.
Authored-by: Juliusz Sompolski <ju...@databricks.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
(cherry picked from commit 0fde146e8676ab9a4aeafebb1684eb7a44660524)
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
.../spark/sql/connect/SimpleSparkConnectService.scala | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/SimpleSparkConnectService.scala b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/SimpleSparkConnectService.scala
index 46f05389ad7..b1376e5131a 100644
--- a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/SimpleSparkConnectService.scala
+++ b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/SimpleSparkConnectService.scala
@@ -38,14 +38,14 @@ private[sql] object SimpleSparkConnectService {
private val stopCommand = "q"
def main(args: Array[String]): Unit = {
+ val conf = new SparkConf()
+ val sparkSession = SparkSession.builder().config(conf).getOrCreate()
+ val sparkContext = sparkSession.sparkContext // init spark context
+ SparkConnectService.start()
+ // scalastyle:off println
+ println("Ready for client connections.")
+ // scalastyle:on println
while (true) {
- val conf = new SparkConf()
- val sparkSession = SparkSession.builder().config(conf).getOrCreate()
- val sparkContext = sparkSession.sparkContext // init spark context
- SparkConnectService.start()
- // scalastyle:off println
- println("Ready for client connections.")
- // scalastyle:on println
val code = StdIn.readLine()
if (code == stopCommand) {
// scalastyle:off println
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org