You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/22 06:56:24 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
dongjoon-hyun opened a new pull request #32288:
URL: https://github.com/apache/spark/pull/32288
### What changes were proposed in this pull request?
This PR aims to support driver-owned on-demand PVC. It means dynamically created PVCs will have the `ownerReference` to `driver` pod.
### Why are the changes needed?
This allows K8s backend scheduler can reuse this later.
### Does this PR introduce _any_ user-facing change?
No. (The default is `false`)
### How was this patch tested?
Manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825187745
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137826/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824598338
**[Test build #137787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137787/testReport)** for PR 32288 at commit [`5da3db1`](https://github.com/apache/spark/commit/5da3db147fdc99e70583d5152345dc2064ff3c99).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825276671
Thank you, @viirya . Merged to master for Apache Spark 3.2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824634566
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137787/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825187745
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137826/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824634566
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137787/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825200976
> This allows K8s backend scheduler can reuse this later.
So based on my understanding, it means that once an executor is removed, the PVC can be still there and reused by the driver on other executor added later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824681116
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42316/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #32288:
URL: https://github.com/apache/spark/pull/32288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824681116
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42316/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825154255
**[Test build #137826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137826/testReport)** for PR 32288 at commit [`ca6efd6`](https://github.com/apache/spark/commit/ca6efd68243ac602f7978b452f1f887f4e43bd93).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825197717
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137828/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #32288:
URL: https://github.com/apache/spark/pull/32288#discussion_r618756176
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
##########
@@ -339,6 +339,9 @@ private[spark] class ExecutorPodsAllocator(
resources
.filter(_.getKind == "PersistentVolumeClaim")
.foreach { resource =>
+ if (conf.get(KUBERNETES_DRIVER_OWN_PVC) && driverPod.nonEmpty) {
+ addOwnerReference(driverPod.get, Seq(resource))
+ }
Review comment:
This overrides the added owner reference at L338 `addOwnerReference(createdExecutorPod, resources)`, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825197493
**[Test build #137828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137828/testReport)** for PR 32288 at commit [`b350f25`](https://github.com/apache/spark/commit/b350f2500e222244b41351fe6d68437f896bb1e1).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825189649
**[Test build #137828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137828/testReport)** for PR 32288 at commit [`b350f25`](https://github.com/apache/spark/commit/b350f2500e222244b41351fe6d68437f896bb1e1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825211296
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #32288:
URL: https://github.com/apache/spark/pull/32288#discussion_r643666101
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
##########
@@ -85,6 +85,7 @@ private[spark] class MountVolumesFeatureStep(conf: KubernetesConf)
.withApiVersion("v1")
.withNewMetadata()
.withName(claimName)
+ .addToLabels(SPARK_APP_ID_LABEL, conf.sparkConf.getAppId)
Review comment:
Yes, `Spark Driver` pod is already launched in the K8s and the driver is building executor pod specs here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825154255
**[Test build #137826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137826/testReport)** for PR 32288 at commit [`ca6efd6`](https://github.com/apache/spark/commit/ca6efd68243ac602f7978b452f1f887f4e43bd93).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825197717
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137828/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824610489
**[Test build #137787 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137787/testReport)** for PR 32288 at commit [`5da3db1`](https://github.com/apache/spark/commit/5da3db147fdc99e70583d5152345dc2064ff3c99).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824598338
**[Test build #137787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137787/testReport)** for PR 32288 at commit [`5da3db1`](https://github.com/apache/spark/commit/5da3db147fdc99e70583d5152345dc2064ff3c99).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #32288:
URL: https://github.com/apache/spark/pull/32288#discussion_r618826955
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
##########
@@ -134,6 +134,13 @@ private[spark] class KubernetesClusterSchedulerBackend(
}
}
+ Utils.tryLogNonFatalError {
+ kubernetesClient
+ .persistentVolumeClaims()
+ .withLabel(SPARK_APP_ID_LABEL, applicationId())
+ .delete()
+ }
Review comment:
Previously, the lifecycle is tied with the executor pod.
Now, the lifecycle is tied with the driver pod. So, it will be deleted when the driver pod die.
This code is to support early deletion at the app termination.
This is the same one for `spark.kubernetes.driver.service.deleteOnTermination`~
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala
##########
@@ -339,6 +339,9 @@ private[spark] class ExecutorPodsAllocator(
resources
.filter(_.getKind == "PersistentVolumeClaim")
.foreach { resource =>
+ if (conf.get(KUBERNETES_DRIVER_OWN_PVC) && driverPod.nonEmpty) {
+ addOwnerReference(driverPod.get, Seq(resource))
+ }
Review comment:
Yes, correct!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825188492
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42356/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #32288:
URL: https://github.com/apache/spark/pull/32288#discussion_r618754117
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
##########
@@ -134,6 +134,13 @@ private[spark] class KubernetesClusterSchedulerBackend(
}
}
+ Utils.tryLogNonFatalError {
+ kubernetesClient
+ .persistentVolumeClaims()
+ .withLabel(SPARK_APP_ID_LABEL, applicationId())
+ .delete()
+ }
Review comment:
So we don't delete PVC before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825189649
**[Test build #137828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137828/testReport)** for PR 32288 at commit [`b350f25`](https://github.com/apache/spark/commit/b350f2500e222244b41351fe6d68437f896bb1e1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-824681076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825216207
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42358/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825198732
Could you review this please, @viirya ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825188455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] mridulm commented on a change in pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
mridulm commented on a change in pull request #32288:
URL: https://github.com/apache/spark/pull/32288#discussion_r643631948
##########
File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
##########
@@ -85,6 +85,7 @@ private[spark] class MountVolumesFeatureStep(conf: KubernetesConf)
.withApiVersion("v1")
.withNewMetadata()
.withName(claimName)
+ .addToLabels(SPARK_APP_ID_LABEL, conf.sparkConf.getAppId)
Review comment:
I am trying to understand this ... will the sparkConf.getAppId be available at this point ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825164686
**[Test build #137826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137826/testReport)** for PR 32288 at commit [`ca6efd6`](https://github.com/apache/spark/commit/ca6efd68243ac602f7978b452f1f887f4e43bd93).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825188492
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42356/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32288: [SPARK-35182][K8S] Support driver-owned on-demand PVC
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32288:
URL: https://github.com/apache/spark/pull/32288#issuecomment-825216207
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42358/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org