[GitHub] [spark] jkleckner commented on pull request #29496: [SPARK-24266][K8S][2.4] Restart the watcher when we receive a version changed from k8s

[GitBox <gi...@apache.org>]

jkleckner commented on pull request #29496:
URL: https://github.com/apache/spark/pull/29496#issuecomment-685104201


   @shockdm This is an attempt to backport the master branch fix which has that behavior [1].
   Also this works fine in our environment for a while now.
   As mentioned above [2], we are using further than v2.4.6 [3] since we wanted to bring in as many fixes as possible.
   
   We launch using the spark-operator-on-k8s [4] mechanism with this version [5] and have not tried any other launch method.
   So it is quite possible that there are issues with directly launching - how do you launch?
   
   [1] https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/LoggingPodStatusWatcher.scala#L114
   [2] https://github.com/apache/spark/pull/29496#issuecomment-682251482
   [3] https://gitlab.com/jkleckner/spark/-/tags/v2.4.7-cphy1
   [4] https://github.com/GoogleCloudPlatform/spark-on-k8s-operator
   [5] https://gitlab.com/jkleckner/spark-on-k8s-operator/-/commit/f30dd1535220813315eea282a3d868aeb14cacb3


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org