You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@celeborn.apache.org by re...@apache.org on 2026/10/12 14:23:12 UTC
(celeborn) branch main updated: [CELEBORN-2474] Send WWW-Authenticate challenge when no auth header present
This is an automated email from the ASF dual-hosted git repository.
RexXiong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 9a088699ba [CELEBORN-2474] Send WWW-Authenticate challenge when no auth header present
9a088699ba is described below
commit 9a088699ba1689136c4247947176de29fb6b74fb
Author: senthh <se...@acceldata.io>
AuthorDate: Mon Oct 12 22:23:00 2026 +0800
[CELEBORN-2474] Send WWW-Authenticate challenge when no auth header present
### What changes were proposed in this pull request?
When `AuthenticationFilter.doFilter` receives a request with no `Authorization` header (the first request in a SPNEGO/Kerberos handshake), respond `401` with `WWW-Authenticate` headers listing all registered auth schemes, so a SPNEGO client (e.g. `curl --negotiate`) can receive the challenge and respond with its Kerberos token. If an `Authorization` header is present but matches no handler, the original error message is preserved.
Also fix the scheme name in the `supportedSchemes` documentation (`celeborn.master.http.auth.supportedSchemes` / `celeborn.worker.http.auth.supportedSchemes`): the Kerberos scheme was listed as `SPNEGO` but the enumeration value is `NEGOTIATE`. Updated `CelebornConf.scala`, `docs/configuration/master.md` and `docs/configuration/worker.md` for consistency with the `ConfigurationSuite` golden-file check.
Files:
- `service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala`
- `common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala`
- `docs/configuration/master.md`
- `docs/configuration/worker.md`
### Why are the changes needed?
Previously the filter returned a bare `401` with no `WWW-Authenticate` header, so the SPNEGO handshake was broken at the very first step (error: "No auth scheme matched for null"). A SPNEGO client never receives the challenge and cannot send its Kerberos token.
### Does this PR resolve a correctness bug?
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
- [ ] Yes
### How was this patch tested?
Verified with `curl --negotiate` against a Celeborn server configured with:
- `celeborn.master.http.auth.supportedSchemes NEGOTIATE`
- `celeborn.master.http.spnego.keytab /path/to/keytab`
- `celeborn.master.http.spnego.principal HTTP/hostREALM`
Before: `401` without `WWW-Authenticate` header, curl never sends the token.
After: `401` with `WWW-Authenticate: NEGOTIATE`, curl completes the SPNEGO handshake and returns `200`.
Closes #3863 from senthh/CELEBORN-9999.
Authored-by: senthh <se...@acceldata.io>
Signed-off-by: Shuang <lv...@alibaba-inc.com>
---
.../org/apache/celeborn/common/CelebornConf.scala | 4 ++--
docs/configuration/master.md | 2 +-
docs/configuration/worker.md | 2 +-
.../server/common/http/HttpAuthUtils.scala | 16 ++++++++++++++
.../http/authentication/AuthenticationFilter.scala | 24 +++++++++++++++++----
.../BasicAuthenticationHandler.scala | 4 +++-
.../http/ApiBaseResourceAuthenticationSuite.scala | 25 ++++++++++++++++++++++
7 files changed, 68 insertions(+), 9 deletions(-)
diff --git a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
index 3eef83205e..3391e3d169 100644
--- a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
@@ -2655,7 +2655,7 @@ object CelebornConf extends Logging {
.version("0.6.0")
.doc("A comma-separated list of master http auth supported schemes." +
"<ul>" +
- " <li>SPNEGO: Kerberos/GSSAPI authentication.</li>" +
+ " <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li>" +
" <li>BASIC: User-defined password authentication, the concreted implementation is" +
" configurable via `celeborn.master.http.auth.basic.provider`.</li>" +
" <li>BEARER: User-defined bearer token authentication, the concreted implementation is" +
@@ -3649,7 +3649,7 @@ object CelebornConf extends Logging {
.version("0.6.0")
.doc("A comma-separated list of worker http auth supported schemes." +
"<ul>" +
- " <li>SPNEGO: Kerberos/GSSAPI authentication.</li>" +
+ " <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li>" +
" <li>BASIC: User-defined password authentication, the concreted implementation is" +
" configurable via `celeborn.worker.http.auth.basic.provider`.</li>" +
" <li>BEARER: User-defined bearer token authentication, the concreted implementation is" +
diff --git a/docs/configuration/master.md b/docs/configuration/master.md
index 86a4158668..8e5b27c3f0 100644
--- a/docs/configuration/master.md
+++ b/docs/configuration/master.md
@@ -53,7 +53,7 @@ license: |
| celeborn.master.http.auth.administers | | false | A comma-separated list of users who have admin privileges, Note, when celeborn.master.http.auth.supportedSchemes is not set, everyone is treated as administrator. | 0.6.0 | |
| celeborn.master.http.auth.basic.provider | org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl | false | User-defined password authentication implementation of org.apache.celeborn.spi.authentication.PasswdAuthenticationProvider | 0.6.0 | |
| celeborn.master.http.auth.bearer.provider | org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl | false | User-defined token authentication implementation of org.apache.celeborn.spi.authentication.TokenAuthenticationProvider | 0.6.0 | |
-| celeborn.master.http.auth.supportedSchemes | | false | A comma-separated list of master http auth supported schemes.<ul> <li>SPNEGO: Kerberos/GSSAPI authentication.</li> <li>BASIC: User-defined password authentication, the concreted implementation is configurable via `celeborn.master.http.auth.basic.provider`.</li> <li>BEARER: User-defined bearer token authentication, the concreted implementation is configurable via `celeborn.master.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
+| celeborn.master.http.auth.supportedSchemes | | false | A comma-separated list of master http auth supported schemes.<ul> <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li> <li>BASIC: User-defined password authentication, the concreted implementation is configurable via `celeborn.master.http.auth.basic.provider`.</li> <li>BEARER: User-defined bearer token authentication, the concreted implementation is configurable via `celeborn.master.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
| celeborn.master.http.host | <localhost> | false | Master's http host. | 0.4.0 | celeborn.metrics.master.prometheus.host,celeborn.master.metrics.prometheus.host |
| celeborn.master.http.idleTimeout | 30s | false | Master http server idle timeout. | 0.5.0 | |
| celeborn.master.http.maxWorkerThreads | 200 | false | Maximum number of threads in the master http worker thread pool. | 0.5.0 | |
diff --git a/docs/configuration/worker.md b/docs/configuration/worker.md
index f33a169936..dd0aa15a55 100644
--- a/docs/configuration/worker.md
+++ b/docs/configuration/worker.md
@@ -118,7 +118,7 @@ license: |
| celeborn.worker.http.auth.administers | | false | A comma-separated list of users who have admin privileges, Note, when celeborn.worker.http.auth.supportedSchemes is not set, everyone is treated as administrator. | 0.6.0 | |
| celeborn.worker.http.auth.basic.provider | org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl | false | User-defined password authentication implementation of org.apache.celeborn.common.authentication.PasswdAuthenticationProvider | 0.6.0 | |
| celeborn.worker.http.auth.bearer.provider | org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl | false | User-defined token authentication implementation of org.apache.celeborn.common.authentication.TokenAuthenticationProvider | 0.6.0 | |
-| celeborn.worker.http.auth.supportedSchemes | | false | A comma-separated list of worker http auth supported schemes.<ul> <li>SPNEGO: Kerberos/GSSAPI authentication.</li> <li>BASIC: User-defined password authentication, the concreted implementation is configurable via `celeborn.worker.http.auth.basic.provider`.</li> <li>BEARER: User-defined bearer token authentication, the concreted implementation is configurable via `celeborn.worker.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
+| celeborn.worker.http.auth.supportedSchemes | | false | A comma-separated list of worker http auth supported schemes.<ul> <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li> <li>BASIC: User-defined password authentication, the concreted implementation is configurable via `celeborn.worker.http.auth.basic.provider`.</li> <li>BEARER: User-defined bearer token authentication, the concreted implementation is configurable via `celeborn.worker.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
| celeborn.worker.http.host | <localhost> | false | Worker's http host. | 0.4.0 | celeborn.metrics.worker.prometheus.host,celeborn.worker.metrics.prometheus.host |
| celeborn.worker.http.idleTimeout | 30s | false | Worker http server idle timeout. | 0.5.0 | |
| celeborn.worker.http.maxWorkerThreads | 200 | false | Maximum number of threads in the worker http worker thread pool. | 0.5.0 | |
diff --git a/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala b/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
index 61d028bee9..47072c9d19 100644
--- a/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
+++ b/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
@@ -21,6 +21,7 @@ import java.util.{Map => JMap}
import scala.collection.JavaConverters._
+import org.apache.celeborn.common.authentication.HttpAuthSchemes
import org.apache.celeborn.server.common.http.authentication.AuthenticationFilter
import org.apache.celeborn.spi.authentication.Credential
@@ -29,6 +30,21 @@ object HttpAuthUtils {
val WWW_AUTHENTICATE_HEADER = "WWW-Authenticate"
// HTTP header used by the client endpoint during an authentication sequence.
val AUTHORIZATION_HEADER = "Authorization"
+ // Realm advertised in the Basic authentication challenge (RFC 7617 requires a realm).
+ val BASIC_AUTH_REALM = "Celeborn"
+
+ /**
+ * Build the value of a single WWW-Authenticate challenge header for an auth scheme.
+ *
+ * The scheme token is case-insensitive on the wire, so we emit the enumeration name (e.g.
+ * NEGOTIATE / BEARER) for most schemes, but RFC 7617 mandates a realm for Basic, so BASIC is
+ * emitted with one: `Basic realm="Celeborn"`. This is the challenge a client (curl, HTTP
+ * auth handlers) uses to decide it can start an auth handshake.
+ */
+ def challengeHeaderValue(scheme: HttpAuthSchemes.HttpAuthScheme): String = scheme match {
+ case HttpAuthSchemes.BASIC => s"""Basic realm="$BASIC_AUTH_REALM""""
+ case _ => scheme.toString
+ }
def getCredentialExtraInfo: JMap[String, String] = {
Map(Credential.CLIENT_IP_KEY ->
diff --git a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
index e0a9a8bbac..99d9cee6e8 100644
--- a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
+++ b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
@@ -30,6 +30,7 @@ import org.apache.celeborn.common.authentication.HttpAuthSchemes
import org.apache.celeborn.common.authentication.HttpAuthSchemes.HttpAuthScheme
import org.apache.celeborn.common.internal.Logging
import org.apache.celeborn.server.common.Service
+import org.apache.celeborn.server.common.http.HttpAuthUtils
import org.apache.celeborn.server.common.http.HttpAuthUtils.AUTHORIZATION_HEADER
import org.apache.celeborn.server.common.http.RestAuditLogger
@@ -162,10 +163,25 @@ class AuthenticationFilter(conf: CelebornConf, serviceName: String) extends Filt
try {
if (matchedHandler == null) {
logDebug(s"No auth scheme matched for url: ${httpRequest.getRequestURL}")
- httpResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
- httpResponse.sendError(
- HttpServletResponse.SC_UNAUTHORIZED,
- s"No auth scheme matched for $authorization")
+ if (authorization == null || authorization.isEmpty) {
+ // First request with no Authorization header. Send WWW-Authenticate
+ // challenge(s) to initiate auth handshake (e.g. SPNEGO/Kerberos
+ // requires a 401+WWW-Authenticate to start the handshake).
+ authSchemeHandlers.keys.foreach { scheme =>
+ httpResponse.addHeader(
+ HttpAuthUtils.WWW_AUTHENTICATE_HEADER,
+ HttpAuthUtils.challengeHeaderValue(scheme))
+ }
+ httpResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
+ httpResponse.setContentType("text/html;charset=iso-8859-1")
+ httpResponse.getWriter.write(
+ "<html><body><h2>HTTP ERROR 401 Unauthorized</h2></body></html>")
+ httpResponse.getWriter.flush
+ } else {
+ httpResponse.sendError(
+ HttpServletResponse.SC_UNAUTHORIZED,
+ s"No auth scheme matched for $authorization")
+ }
} else {
HTTP_AUTH_TYPE.set(matchedHandler.authScheme.toString)
HTTP_CLIENT_IDENTIFIER.set(matchedHandler.authenticate(httpRequest, httpResponse))
diff --git a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
index e5d8ba5399..6d17fe6e0a 100644
--- a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
+++ b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
@@ -81,7 +81,9 @@ class BasicAuthenticationHandler(providerClass: String) extends AuthenticationHa
authUser = creds.take(1).headOption.filterNot(_.isEmpty).getOrElse("anonymous")
} else {
if (creds.size < 2 || creds(0).trim.isEmpty || creds(1).trim.isEmpty) {
- response.setHeader(WWW_AUTHENTICATE_HEADER, authScheme.toString)
+ response.setHeader(
+ WWW_AUTHENTICATE_HEADER,
+ HttpAuthUtils.challengeHeaderValue(authScheme))
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
} else {
val Seq(user, password) = creds.toSeq.take(2)
diff --git a/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala b/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
index 8fb95c7ef7..677b6b1707 100644
--- a/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
+++ b/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
@@ -22,6 +22,8 @@ import java.util.Base64
import javax.servlet.http.HttpServletResponse
import javax.ws.rs.core.MediaType
+import scala.collection.JavaConverters._
+
import org.apache.celeborn.common.CelebornConf
import org.apache.celeborn.common.authentication.HttpAuthSchemes
import org.apache.celeborn.common.network.TestHelper
@@ -151,4 +153,27 @@ abstract class ApiBaseResourceAuthenticationSuite extends HttpTestHelper {
}
}
}
+
+ test("request without Authorization header gets 401 with WWW-Authenticate challenges") {
+ // The suite configures BASIC + BEARER schemes; a request with no Authorization header must be
+ // challenged with one WWW-Authenticate value per registered scheme. The Basic challenge carries
+ // a realm (RFC 7617), other schemes use their bare token.
+ val response = webTarget.path("shuffle").request(MediaType.TEXT_PLAIN).get()
+ assert(HttpServletResponse.SC_UNAUTHORIZED == response.getStatus)
+
+ val challenges =
+ response.getStringHeaders.get(HttpAuthUtils.WWW_AUTHENTICATE_HEADER).asScala.toSet
+ assert(challenges.contains("Basic realm=\"Celeborn\""))
+ assert(challenges.contains(HttpAuthSchemes.BEARER.toString))
+ }
+
+ test("request with unmatched Authorization scheme keeps 401 and original error") {
+ // An Authorization header that matches no configured handler must still 401 and must NOT be
+ // challenged (no new WWW-Authenticate added for it).
+ val response = webTarget.path("shuffle")
+ .request(MediaType.TEXT_PLAIN)
+ .header(AUTHORIZATION_HEADER, "Digest realm=\"x\"")
+ .get()
+ assert(HttpServletResponse.SC_UNAUTHORIZED == response.getStatus)
+ }
}