You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2026/02/24 07:06:37 UTC

[PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executors [spark]

dongjoon-hyun opened a new pull request, #54442:
URL: https://github.com/apache/spark/pull/54442

   ### What changes were proposed in this pull request?
   
   This PR aims to support `NetworkPolicy` for Spark executors.
   
   ### Why are the changes needed?
   
   `NetworkPolicy` is frequently used in the production to isolate Spark applications.
   
   - https://kubernetes.io/docs/concepts/services-networking/network-policies/
   
   ### Does this PR introduce _any_ user-facing change?
   
   This is a security feature to make Spark K8s executor pods access only from the pods with the same application ID.
   
   ### How was this patch tested?
   
   Pass the CIs with the newly added test suite.
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: `Gemini 3.1 Pro (High)` on `Antigravity`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executor pods [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #54442: [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executor pods
URL: https://github.com/apache/spark/pull/54442


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executors pods [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #54442:
URL: https://github.com/apache/spark/pull/54442#issuecomment-3949705462

   cc @peter-toth 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executors pods [spark]

Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on PR #54442:
URL: https://github.com/apache/spark/pull/54442#issuecomment-3949974475

   Does this change need to be explicitly mentioned in migration docs?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executor pods [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #54442:
URL: https://github.com/apache/spark/pull/54442#issuecomment-3952669543

   Thank you, @peter-toth . I addressed your comment by updating the migration guide.
   > - Since Spark 4.2, Spark configures a `NetworkPolicy` by default so that executor pods only accept ingress traffic from the driver and peer executors within the same job. To disable this and restore the legacy behavior, set `spark.kubernetes.driver.pod.excludedFeatureSteps` to `org.apache.spark.deploy.k8s.features.NetworkPolicyFeatureStep`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-55653][K8S] Support `NetworkPolicy` for Spark executor pods [spark]

Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #54442:
URL: https://github.com/apache/spark/pull/54442#issuecomment-3952685273

   Merged to main


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org