You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by jinxing64 <gi...@git.apache.org> on 2018/03/09 05:55:10 UTC
[GitHub] spark pull request #20781: [SPARK-23637][YARN]Yarn might allocate more resou...
GitHub user jinxing64 opened a pull request:
https://github.com/apache/spark/pull/20781
[SPARK-23637][YARN]Yarn might allocate more resource if a same executor is killed multiple times.
## What changes were proposed in this pull request?
`YarnAllocator` uses `numExecutorsRunning` to track the number of running executor. `numExecutorsRunning` is used to check if there're executors missing and need to allocate more.
In current code, `numExecutorsRunning` can be negative when driver asks to kill a same idle executor multiple times.
## How was this patch tested?
UT added
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinxing64/spark SPARK-23637
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20781.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20781
----
commit bd6f8a1dd608d95f6faf2758cbd36e6328f48eb3
Author: jinxing <ji...@...>
Date: 2018-03-09T05:53:10Z
[SPARK-23637][YARN]Yarn might allocate more resource if a same executor is killed multiple times.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
This basically means that drive send multiple same kill requests to AM, right? I'm wondering how this would happen, shall we also guarantee this in the driver side?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
Still I'm not so sure about the root cause, but adding defensive code seems no harm.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
Since the change for `YarnAllocator: killExecutor` is easy. Do you think it's worth to have this defense?
Thanks again for review.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
cc @vanzin @tgravescs @cloud-fan @djvulee
Could you please help review this ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@vanzin
Thanks for review~
1. I spent some time but didn't find the reason why same executor is killed multiple times and I cannot reproduce either.
2. I found that same completed container can be processed multiple times. It happens now and then. Seems yarn doesn't promise that same completed container only returned in one response (https://github.com/apache/spark/blob/master/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala#L268)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88178/testReport)** for PR 20781 at commit [`049ed49`](https://github.com/apache/spark/commit/049ed495bb8c7da05f034300237cf9360bc0848d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88837/testReport)** for PR 20781 at commit [`049ed49`](https://github.com/apache/spark/commit/049ed495bb8c7da05f034300237cf9360bc0848d).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao
Thanks for advice. I spent some time digging to find why multiple `kill` sent from Driver to AM, but didn't figure out a way to reproduce.
I come to find that it's possible `YarnAllocator` process same completed container multiple times(https://github.com/apache/spark/blob/master/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala#L573 this log is printed multiple times for same container). And I made another change(see last commit in this pr) to propose my idea -- replace `numExecutorsRunning` with a set.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88116/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88127/testReport)** for PR 20781 at commit [`a177a63`](https://github.com/apache/spark/commit/a177a63fa9caf0228cad8d35892d9501d763698a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88837/testReport)** for PR 20781 at commit [`049ed49`](https://github.com/apache/spark/commit/049ed495bb8c7da05f034300237cf9360bc0848d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88178/testReport)** for PR 20781 at commit [`049ed49`](https://github.com/apache/spark/commit/049ed495bb8c7da05f034300237cf9360bc0848d).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao
Thanks again for review.
It does exist in my cluster that same container can be processed multiple times, which will make `numExecutorsRunning` negative. I think I've ever find this issue in another Spark jira, but I cannot find it now.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88127/testReport)** for PR 20781 at commit [`a177a63`](https://github.com/apache/spark/commit/a177a63fa9caf0228cad8d35892d9501d763698a).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao Thanks for taking look.
Yes, it does happen. we have jobs which have already finished all the tasks but still holding 40~100 executors.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
Merging to master / 2.3.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1428/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88178/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88116/testReport)** for PR 20781 at commit [`bd6f8a1`](https://github.com/apache/spark/commit/bd6f8a1dd608d95f6faf2758cbd36e6328f48eb3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20781: [SPARK-23637][YARN]Yarn might allocate more resou...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/20781#discussion_r173709533
--- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -81,7 +81,7 @@ private[yarn] class YarnAllocator(
private val releasedContainers = Collections.newSetFromMap[ContainerId](
new ConcurrentHashMap[ContainerId, java.lang.Boolean])
- private val numExecutorsRunning = new AtomicInteger(0)
+ private val runningExecutors = new java.util.concurrent.ConcurrentHashMap[String, Unit]()
--- End diff --
This can be changed to `Collections.newSetFromMap`, since we only need `Set` instead of `Map`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88116/testReport)** for PR 20781 at commit [`bd6f8a1`](https://github.com/apache/spark/commit/bd6f8a1dd608d95f6faf2758cbd36e6328f48eb3).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1466/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
retest this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1422/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
The change looks good, but did you look at why the code is trying to kill the same executor multiple times? That sounds like it could be a possible bug on the scheduler backend, which should be keeping track of these things.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jinxing64 <gi...@git.apache.org>.
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@vanzin Thanks for merging.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by jerryshao <gi...@git.apache.org>.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
Does it happen only in dynamic allocation enabled scenario?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20781: [SPARK-23637][YARN]Yarn might allocate more resou...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20781
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1912/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88127/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88837/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20781: [SPARK-23637][YARN]Yarn might allocate more resource if ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org