You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "j1wonpark (via GitHub)" <gi...@apache.org> on 2026/06/09 06:43:16 UTC
[PR] [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header [spark]
j1wonpark opened a new pull request, #56389:
URL: https://github.com/apache/spark/pull/56389
### What changes were proposed in this pull request?
The Scala/JVM Spark Connect client sends the access token (from the `token` connection param) in a non-standard `Authentication` gRPC metadata header. This changes it to the standard `Authorization`, and adds a unit test.
```scala
- Metadata.Key.of("Authentication", Metadata.ASCII_STRING_MARSHALLER)
+ Metadata.Key.of("Authorization", Metadata.ASCII_STRING_MARSHALLER)
```
### Why are the changes needed?
Bearer tokens belong in `Authorization` (RFC 6750). The old key breaks token auth and is inconsistent with every other consumer:
- server-side `PreSharedKeyAuthenticationInterceptor` reads the token from `Authorization`;
- the local-channel path in the same client already uses `Authorization`;
- the Python client uses gRPC's `access_token_call_credentials` (i.e. `Authorization`);
- standard JWT proxies (e.g. Envoy/Istio) extract the bearer from `Authorization` and otherwise reject the request.
The typo dates to SPARK-42533 (2023) and went unnoticed until an in-tree server consumer (`PreSharedKeyAuthenticationInterceptor`, SPARK-57066) appeared.
### Does this PR introduce _any_ user-facing change?
Yes. The Scala/JVM client now sends the token in `Authorization` instead of `Authentication`, fixing token auth against the pre-shared-key interceptor and standard JWT proxies.
### How was this patch tested?
New unit test in `SparkConnectClientSuite` asserting `AccessTokenCallCredentials` emits `Authorization: Bearer <token>` (RED before the fix, GREEN after); full suite passes.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56389:
URL: https://github.com/apache/spark/pull/56389#issuecomment-4744953642
Merged to master/4.x/4.2/4.1/4.0.
Also, cc @huaxingao because this was a security issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header [spark]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #56389:
URL: https://github.com/apache/spark/pull/56389#issuecomment-4668068374
this seems to be a bug, cc @HyukjinKwon @hvanhovell @Kimahriman
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56389:
URL: https://github.com/apache/spark/pull/56389#issuecomment-4744722489
IIUC, the feature works since 3.4.1 ~ 3.5.8, and seems to ben broken at Apache Spark 4.0.0 with the following.
- https://github.com/apache/spark/pull/50006
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #56389: [SPARK-57336][CONNECT] Send the bearer token in the standard Authorization header
URL: https://github.com/apache/spark/pull/56389
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org