You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2026/09/11 17:02:54 UTC
[PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]
viirya opened a new pull request, #24:
URL: https://github.com/apache/spark-connect-gateway/pull/24
The e2e workflow now covers baseline routing, JWT identity, the backend
trust boundary, and per-tenant pool isolation, but nothing in CI stands
the gateway up as more than one replica. The multi-replica walkthrough
under deploy/examples/e2e-multi-replica-redis has never run
automatically, so the chart's Redis topology and the gateway's behaviour
when a replica dies are only verified by hand.
The cross-replica semantics themselves are already covered in-process by
the ha_smoke example, which runs in the integration job against a real
Redis. What is missing is the Kubernetes shape around it: the bundled
Redis StatefulSet, the synthesized redis:// URL, two gateway replicas
behind one Service, and a pod actually being deleted.
This adds an e2e-multi-replica-redis job that asserts:
(1) Both gateway replicas report affinity_store=redis on startup. The
pod count is checked first, so a selector that matched nothing could not
let the check pass silently.
(2) A session driven through the Service writes its binding to Redis,
under the documented key shape, with a value that is one of the real
backend pod IPs and a TTL within the configured hour. Redis is asserted
empty beforehand so a stale key cannot satisfy this.
(3) After deleting one replica, the binding is unchanged, and a client
arriving at the replacement pod -- port-forwarded directly, so the
surviving replica cannot serve it -- reaches the same backend. The
replacement's own log must name that backend, an address it could only
have read from Redis.
(4) Fresh sessions still spread across both backends. Without this,
assertion 3 would also pass on a gateway that always picked the same
backend regardless of affinity.
Verified locally against a kind cluster first. The replacement replica
returned the same serverSideSessionId as the original, confirming the
same Spark driver answered, and eight fresh sessions split 4/3 across the
two backends while the pinned session stayed put.
---
Verified locally against a kind cluster (`scg-redis`) before writing the workflow.
The decisive evidence for assertion 3: after deleting the replica that had served
the session, a client port-forwarded **directly to the replacement pod** got back
```
serverSideSessionId: 28870e4e-f886-42b9-b686-04a095e0abb9
```
byte-identical to what the original replica returned. That id is minted by the
Spark driver, so the same driver answered — and the replacement pod, which had
never seen the session, could only have found that backend in Redis.
The negative control (assertion 4) split 8 fresh sessions 4/3 across the two
backends while the pinned session stayed on its own, so the pinning is a property
of the binding rather than of the pool.
Cross-replica *semantics* are already covered by `ha_smoke` in the integration
job; this job covers the Kubernetes topology around it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya closed pull request #24: [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI
URL: https://github.com/apache/spark-connect-gateway/pull/24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #24:
URL: https://github.com/apache/spark-connect-gateway/pull/24#issuecomment-5657550555
Thanks @dongjoon-hyun !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org