You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2025/02/03 17:33:35 UTC
(spark) branch branch-4.0 updated: [SPARK-51061][CORE] Hide `Jetty` info in REST Submission API
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 6cd1a9a7206b [SPARK-51061][CORE] Hide `Jetty` info in REST Submission API
6cd1a9a7206b is described below
commit 6cd1a9a7206b541743135fd25e74d049aa4f785e
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Mon Feb 3 09:33:16 2025 -0800
[SPARK-51061][CORE] Hide `Jetty` info in REST Submission API
### What changes were proposed in this pull request?
This PR aims to hide `Jetty` info in REST Submission API like SPARK-46239.
- #44158
### Why are the changes needed?
Start `Spark Master` server.
```
$ SPARK_MASTER_OPTS='-Dspark.master.rest.enabled=true' sbin/start-master.sh
```
**BEFORE:**
```
$ curl -vv http://0.0.0.0:6066/v1/submissions/readyz
* Trying 0.0.0.0:6066...
* Connected to 0.0.0.0 (0.0.0.0) port 6066
> GET /v1/submissions/readyz HTTP/1.1
> Host: 0.0.0.0:6066
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Date: Sun, 02 Feb 2025 05:14:04 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 114
< Server: Jetty(11.0.24)
<
{
"action" : "ReadyzResponse",
"message" : "",
"serverSparkVersion" : "4.1.0-SNAPSHOT",
"success" : true
* Connection #0 to host 0.0.0.0 left intact
}
```
**AFTER**
```
$ curl -vv http://0.0.0.0:6066/v1/submissions/readyz
* Trying 0.0.0.0:6066...
* Connected to 0.0.0.0 (0.0.0.0) port 6066
> GET /v1/submissions/readyz HTTP/1.1
> Host: 0.0.0.0:6066
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Date: Sun, 02 Feb 2025 05:12:41 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 114
<
{
"action" : "ReadyzResponse",
"message" : "",
"serverSparkVersion" : "4.1.0-SNAPSHOT",
"success" : true
* Connection #0 to host 0.0.0.0 left intact
}
```
### Does this PR introduce _any_ user-facing change?
No, there is no technical behavior change because this is a meta-data.
### How was this patch tested?
Manual test with the above procedure.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #49762 from dongjoon-hyun/SPARK-51061.
Authored-by: Dongjoon Hyun <do...@apache.org>
Signed-off-by: Dongjoon Hyun <do...@apache.org>
(cherry picked from commit ac8f8b08941bec66710a5d845d3bde84dda4441f)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
.../org/apache/spark/deploy/rest/RestSubmissionServer.scala | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala b/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala
index 9e3aab125689..f8afa86fd36f 100644
--- a/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala
@@ -25,7 +25,7 @@ import scala.io.Source
import com.fasterxml.jackson.core.JsonProcessingException
import jakarta.servlet.DispatcherType
import jakarta.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse}
-import org.eclipse.jetty.server.{HttpConnectionFactory, Server, ServerConnector}
+import org.eclipse.jetty.server.{HttpConfiguration, HttpConnectionFactory, Server, ServerConnector}
import org.eclipse.jetty.servlet.{FilterHolder, ServletContextHandler, ServletHolder}
import org.eclipse.jetty.util.thread.{QueuedThreadPool, ScheduledExecutorScheduler}
import org.json4s._
@@ -104,6 +104,13 @@ private[spark] abstract class RestSubmissionServer(
threadPool.setDaemon(true)
val server = new Server(threadPool)
+ // Hide information.
+ val httpConfig = new HttpConfiguration()
+ logDebug("Using setSendServerVersion: false")
+ httpConfig.setSendServerVersion(false)
+ logDebug("Using setSendXPoweredBy: false")
+ httpConfig.setSendXPoweredBy(false)
+
val connector = new ServerConnector(
server,
null,
@@ -112,7 +119,7 @@ private[spark] abstract class RestSubmissionServer(
null,
-1,
-1,
- new HttpConnectionFactory())
+ new HttpConnectionFactory(httpConfig))
connector.setHost(host)
connector.setPort(startPort)
connector.setReuseAddress(!Utils.isWindows)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org