You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by zs...@apache.org on 2017/11/02 18:06:43 UTC
spark git commit: [SPARK-22243][DSTREAM] spark.yarn.jars should
reload from config when checkpoint recovery
Repository: spark
Updated Branches:
refs/heads/master e3f67a97f -> 882079f5c
[SPARK-22243][DSTREAM] spark.yarn.jars should reload from config when checkpoint recovery
## What changes were proposed in this pull request?
the previous [PR](https://github.com/apache/spark/pull/19469) is deleted by mistake.
the solution is straight forward.
adding "spark.yarn.jars" to propertiesToReload so this property will load from config.
## How was this patch tested?
manual tests
Author: ZouChenjun <zo...@youzan.com>
Closes #19637 from ChenjunZou/checkpoint-yarn-jars.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/882079f5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/882079f5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/882079f5
Branch: refs/heads/master
Commit: 882079f5c6de40913a27873f2fa3306e5d827393
Parents: e3f67a9
Author: ZouChenjun <zo...@youzan.com>
Authored: Thu Nov 2 11:06:37 2017 -0700
Committer: Shixiong Zhu <zs...@gmail.com>
Committed: Thu Nov 2 11:06:37 2017 -0700
----------------------------------------------------------------------
.../src/main/scala/org/apache/spark/streaming/Checkpoint.scala | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/882079f5/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala
----------------------------------------------------------------------
diff --git a/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala b/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala
index 40a0b8e..9ebb91b 100644
--- a/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala
+++ b/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala
@@ -53,6 +53,7 @@ class Checkpoint(ssc: StreamingContext, val checkpointTime: Time)
"spark.driver.host",
"spark.driver.port",
"spark.master",
+ "spark.yarn.jars",
"spark.yarn.keytab",
"spark.yarn.principal",
"spark.yarn.credentials.file",
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org