You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/09 07:16:42 UTC
[GitHub] [spark] grarkydev opened a new pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
grarkydev opened a new pull request #32077:
URL: https://github.com/apache/spark/pull/32077
Co-authored-by: hongdd <ho...@cmss.chinamobile.com>
### What changes were proposed in this pull request?
Supporting `SparkAppHandle` object to be able to manage a running Spark application on Kubernetes. It can be used to monitor the application changes and to stop the application by pod deletion.
This Pull Request has been raised due to a inactivity of a previous one - https://github.com/apache/spark/pull/30520
### Why are the changes needed?
There is an inconsistency in the Spark application managing with `SparkAppHandle` object between Kubernetes and other resource managers such as Yarn/Mesos.
Currently, this feature is not properly implemented on Kubernetes which may cause some issues.
### Does this PR introduce _any_ user-facing change?
Yes, it changes the behavior of `SparkAppHandle` object which the user may use to communicate with the launched Spark application. Its interface is remained as it is. Some missing functionalities have been implemented.
### How was this patch tested?
Few unit tests has been added. May be found in `org.apache.spark.deploy.k8s.submit` package:
- `PodStatusWatcherSuite` - new ones
- `ClientSuite` - added some
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev removed a comment on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev removed a comment on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-816469593
> Hi, @grarkydev . In this case, could you keep the authorship of @hddong , please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev edited a comment on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev edited a comment on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-817521986
> Hi, @grarkydev . In this case, could you keep the authorship of @hddong , please?
@dongjoon-hyun Done. Changed in a commit message and in a PR's description. Should this co-authoring be added somewhere else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32077: [SPARK-33357][Kubernetes] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-814663612
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev commented on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev commented on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-817521986
> Hi, @grarkydev . In this case, could you keep the authorship of @hddong , please?
Done. Changed in a commit message and in a PR's description. Should this co-authoring be added somewhere else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev closed pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev closed pull request #32077:
URL: https://github.com/apache/spark/pull/32077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev closed pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev closed pull request #32077:
URL: https://github.com/apache/spark/pull/32077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev commented on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev commented on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-824593413
Jenkins, retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-814663612
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev commented on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev commented on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-816469593
> Hi, @grarkydev . In this case, could you keep the authorship of @hddong , please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] grarkydev commented on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes
Posted by GitBox <gi...@apache.org>.
grarkydev commented on pull request #32077:
URL: https://github.com/apache/spark/pull/32077#issuecomment-823038349
@dongjoon-hyun any further actions required? Co-authoring has been updated as you requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org