You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/03/06 02:42:23 UTC
(spark) branch branch-3.5 updated: [SPARK-51407][CONNECT][DOCS][3.5] Document missed `Spark Connect` configurations
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new 0b88c4dc48ec [SPARK-51407][CONNECT][DOCS][3.5] Document missed `Spark Connect` configurations
0b88c4dc48ec is described below
commit 0b88c4dc48ecd906af63952e3364ae77402f084e
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Wed Mar 5 18:42:12 2025 -0800
[SPARK-51407][CONNECT][DOCS][3.5] Document missed `Spark Connect` configurations
### What changes were proposed in this pull request?
This PR aims to documents the missed `spark.connect.*` configurations by syncing.
- connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala
- docs/configuration.md
### Why are the changes needed?
**Apache Spark 3.5.0**
- spark.connect.jvmStacktrace.maxSize
- spark.sql.connect.ui.retainedSessions
- spark.sql.connect.ui.retainedStatements
### Does this PR introduce _any_ user-facing change?
This updates only a config documentation and `configuration` HTML page.
### How was this patch tested?
Manual review.
```
$ cd docs
$ bundle install
$ SKIP_API=1 bundle exec jekyll build
```
<img width="1071" alt="Screenshot 2025-03-05 at 16 07 32" src="https://github.com/user-attachments/assets/9f3a69dc-67a5-4107-9105-3b7c81944c2f" />
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #50172 from dongjoon-hyun/SPARK-51407-3.5.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../apache/spark/sql/connect/config/Connect.scala | 1 +
docs/configuration.md | 24 ++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala
index 253ac38f9cf9..cea1f65fef23 100644
--- a/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala
+++ b/connector/connect/server/src/main/scala/org/apache/spark/sql/connect/config/Connect.scala
@@ -26,6 +26,7 @@ object Connect {
val CONNECT_GRPC_BINDING_PORT =
buildStaticConf("spark.connect.grpc.binding.port")
+ .doc("The port for Spark Connect server to bind.")
.version("3.4.0")
.intConf
.createWithDefault(ConnectCommon.CONNECT_GRPC_BINDING_PORT)
diff --git a/docs/configuration.md b/docs/configuration.md
index 302348aa0f0e..f3b29254283b 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -3281,6 +3281,30 @@ Expression types in proto.</td>
Command types in proto.</td>
<td>3.4.0</td>
</tr>
+<tr>
+ <td><code>spark.connect.jvmStacktrace.maxSize</code></td>
+ <td>
+ 2048
+ </td>
+ <td>Sets the maximum stack trace size to display when `spark.sql.pyspark.jvmStacktrace.enabled` is true.</td>
+ <td>3.5.0</td>
+</tr>
+<tr>
+ <td><code>spark.sql.connect.ui.retainedSessions</code></td>
+ <td>
+ 200
+ </td>
+ <td>The number of client sessions kept in the Spark Connect UI history.</td>
+ <td>3.5.0</td>
+</tr>
+<tr>
+ <td><code>spark.sql.connect.ui.retainedStatements</code></td>
+ <td>
+ 200
+ </td>
+ <td>The number of statements kept in the Spark Connect UI history.</td>
+ <td>3.5.0</td>
+</tr>
</table>
### Security
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org