You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "viirya (via GitHub)" <gi...@apache.org> on 2026/09/11 17:02:54 UTC

[PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]

viirya opened a new pull request, #24:
URL: https://github.com/apache/spark-connect-gateway/pull/24

   The e2e workflow now covers baseline routing, JWT identity, the backend
   trust boundary, and per-tenant pool isolation, but nothing in CI stands
   the gateway up as more than one replica. The multi-replica walkthrough
   under deploy/examples/e2e-multi-replica-redis has never run
   automatically, so the chart's Redis topology and the gateway's behaviour
   when a replica dies are only verified by hand.
   
   The cross-replica semantics themselves are already covered in-process by
   the ha_smoke example, which runs in the integration job against a real
   Redis. What is missing is the Kubernetes shape around it: the bundled
   Redis StatefulSet, the synthesized redis:// URL, two gateway replicas
   behind one Service, and a pod actually being deleted.
   
   This adds an e2e-multi-replica-redis job that asserts:
   
   (1) Both gateway replicas report affinity_store=redis on startup. The
   pod count is checked first, so a selector that matched nothing could not
   let the check pass silently.
   
   (2) A session driven through the Service writes its binding to Redis,
   under the documented key shape, with a value that is one of the real
   backend pod IPs and a TTL within the configured hour. Redis is asserted
   empty beforehand so a stale key cannot satisfy this.
   
   (3) After deleting one replica, the binding is unchanged, and a client
   arriving at the replacement pod -- port-forwarded directly, so the
   surviving replica cannot serve it -- reaches the same backend. The
   replacement's own log must name that backend, an address it could only
   have read from Redis.
   
   (4) Fresh sessions still spread across both backends. Without this,
   assertion 3 would also pass on a gateway that always picked the same
   backend regardless of affinity.
   
   Verified locally against a kind cluster first. The replacement replica
   returned the same serverSideSessionId as the original, confirming the
   same Spark driver answered, and eight fresh sessions split 4/3 across the
   two backends while the pinned session stayed put.
   
   ---
   
   Verified locally against a kind cluster (`scg-redis`) before writing the workflow.
   The decisive evidence for assertion 3: after deleting the replica that had served
   the session, a client port-forwarded **directly to the replacement pod** got back
   
   ```
   serverSideSessionId: 28870e4e-f886-42b9-b686-04a095e0abb9
   ```
   
   byte-identical to what the original replica returned. That id is minted by the
   Spark driver, so the same driver answered — and the replacement pod, which had
   never seen the session, could only have found that backend in Redis.
   
   The negative control (assertion 4) split 8 fresh sessions 4/3 across the two
   backends while the pinned session stayed on its own, so the pinning is a property
   of the binding rather than of the pool.
   
   Cross-replica *semantics* are already covered by `ha_smoke` in the integration
   job; this job covers the Kubernetes topology around it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya closed pull request #24: [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI
URL: https://github.com/apache/spark-connect-gateway/pull/24


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-59445] Run the multi-replica Redis affinity end-to-end walkthrough in CI [spark-connect-gateway]

Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on PR #24:
URL: https://github.com/apache/spark-connect-gateway/pull/24#issuecomment-5657550555

   Thanks @dongjoon-hyun !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org