You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by kxepal <gi...@git.apache.org> on 2016/11/21 10:57:50 UTC

[GitHub] spark pull request #15961: [SPARK-18523][PySpark]Make SparkContext.stop more...

GitHub user kxepal opened a pull request:

    https://github.com/apache/spark/pull/15961

    [SPARK-18523][PySpark]Make SparkContext.stop more reliable

    ## What changes were proposed in this pull request?
    
    This PR fixes SparkContext broken state in which it may fall if spark driver get crashed or killed by OOM.
    
    ## How was this patch tested?
    
    1. Start SparkContext;
    2. Find Spark driver process and `kill -9` it;
    3. Call `sc.stop()`;
    4. Create new SparkContext after that;
    
    Without this patch you will crash on step 3 and won't be able to do step 4 without manual reset private attibutes or IPython notebook / shell restart.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kxepal/spark 18523-make-spark-context-stop-more-reliable

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/15961.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #15961
    
----

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @holdenk 
    I was thought about to use warning there, but found that's may useless one. When I do Spark context stop I actually want to achieve either:
    1. Clean up things before exit;
    2. Start new Spark context with different configuration or just to restart broken one;
    
    In both cases I wouldn't care much about underlying JVM process state - I'm shutting down things, it's over, no matter how healthy or broken they are.
    
    Warnings are good to take you attention onto some problem and hint you to make some actions to sole them. Like Spark warns you if you pass unknown key to SparkConfig - it's not fatal, but such kind of warning you can fix.
    
    In our case I can do nothing with this warning. I could only say "oh, ok", but really there no action could be done to solve that problem.
    
    The different case is when JVM process dies in the middle of something, when you're not expect that. Here you'll still get your Py4JError exception with the same not very useful "Connection refused", but in this case you have to take some actions to solve that issue (restart SparkContext, increase driver memory, optimize your code flow etc.). In case of Py4JError on `sc.stop()` much likely you won't have to do anything special and shouldn't.
    
    Let me know what you think about. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @holdenk 
    Agree with you here. The message is fixed, PR rebased.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    cc @joshrosen @zsxwing 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @holdenk 
    Sure, done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    I think that warning gives a false sense of security - as I indicated above this swallows all of the Py4J errors and there are a host of things which could cause the Py4J bridge to break down. The warning text I would choose would be more along the lines of: "Unable to cleanly shutdown Spark JVM process. It is possible that the process has crashed or been killed, but may also be in a zombie state."
    
    This way the end user know to take a look and verify that the previous Spark JVM did indeed exit.
    
    I'm still not sure that we want to do this though - if the Spark JVM has failed stopping and restarting the Spark context is going to have all of the previous references to Java objects invalid (not just the RDDs/DataFrames which happens on a stop/start of the Scala Spark context normally). It seems like the correct action for the user to take when the Py4J bridge breaks is starting over from scratch, either by exiting and re-running their notebook or otherwise re-submitting there job.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #15961: [SPARK-18523][PySpark]Make SparkContext.stop more...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/15961


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    **[Test build #3442 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3442/consoleFull)** for PR 15961 at commit [`b47f12d`](https://github.com/apache/spark/commit/b47f12d45fe6d3ea6ecf45222e5903cb6c188a32).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @holdenk 
    Thanks for a warning message text. Nice one!
    
    > I indicated above this swallows all of the Py4J errors and there are a host of things which could cause the Py4J bridge to break down. 
    
    Suddenly, as you may see in [issue's traceback](https://issues.apache.org/jira/browse/SPARK-18523), it's py4j who raises too general exception on such kind of problem. I was too expected to see there Py4JNetworkError since it's network communication issue, but this didn't happen. The really good exceptions are get swallowed somewhere in the middle and get just printed to stderr via logging, but I'm not sure how to reraise them and how much things this will break.
    
    > It seems like the correct action for the user to take when the Py4J bridge breaks is starting over from scratch, either by exiting and re-running their notebook or otherwise re-submitting there job.
    
    Yes, that's what happens now: in case of failure we have to shutdown notebook, start it and re-run all the cells again. If we're not running in notebook - crash whole the script. Here comes two issues:
    
    1. Usability. If you made some mistake or Spark job eventually fails, you wouldn't restart whole the notebook, but run cell with `sc.stop` and else cleanup stuff and re-run your Spark cells. That's simple procedure. But when Spark context stop fails, you have to follow plan B: restart all the things and re-run all the cells. That's could be quite boring and actually it is.
    
    2. Correctness. SparkContext is a global shared mutable object and if we cannot correctly reset it state to default to start over that feels like something really wrong here. Should we run all the code that uses SparkContext in subprocesses just to be able to implement retry logic otherwise? 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    Thanks!



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    hm you didn't need to do that. It was just a transient issue with github's own api.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    Merging in master/branch-2.1.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @srowen 
    I was thought I pretty well described that above. TL;DR this information is not useful and you cannot do anything with it, but ignore, imho. 
    
    But if you insist, I'll add a warning, that's not a problem. Just want to make sure that this is really reasonable to do.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    I can't think of any code I've seen that would fail to log a failure like this. Why do you say it's so completely unactionable that nobody should know about it? what's the downside to giving this information versus making it impossible to tell that the JVM failed to shut down?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #15961: [SPARK-18523][PySpark]Make SparkContext.stop more...

Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on a diff in the pull request:

    https://github.com/apache/spark/pull/15961#discussion_r89671365
  
    --- Diff: python/pyspark/context.py ---
    @@ -373,8 +375,19 @@ def stop(self):
             Shut down the SparkContext.
             """
             if getattr(self, "_jsc", None):
    -            self._jsc.stop()
    -            self._jsc = None
    +            try:
    +                self._jsc.stop()
    +            except Py4JError:
    +                # Case: SPARK-18523
    +                warnings.warn(
    +                    'Unable to cleanly shutdown Spark JVM process.'
    +                    ' It is possible that the process has crashed'
    +                    ' or been killed, but may also be in a zombie state.',
    --- End diff --
    
    sgtm either way :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @holdenk @srowen 
    Added a warning message, please take a second look.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    This looks good to me pending @rxins proposed wording change. I think restarting a stopped SparkContext which has been killed by an OOM killer or other issue is probably not a good thing to generally do and the warning text makes it clear enough that the machine may be in a bad state so the user can investigate if necessary.
    
    While I'm unlikely to use this, it sounds like it could make life easier for some notebook users to avoid having to restart their notebook kernel. Thanks for taking the time on this PR @kxepal :)
    
    Note: that if you do have automatic retry logic that likely should not be implemented this way - there is a chance this might lead to a very bad state (hence the warning message). [Just mentioning this since it was mentioned as one of the possible uses in our discussion].


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    I don't see why you wouldn't warn()? otherwise there's no indication that stop() failed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    I'm not sure we want to silently swallow all Py4J errors on `sc.stop()` - at least we should probably warn about the situation (since it seems like it might be possible the JVM process is stuck or another issue has broken the py4j bridge).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    @srowen 
    
    > Why do you say it's so completely unactionable that nobody should know about it? 
    
    That's just from my experience: in all the cases when drivers dies, it dies in the middle of something, when you do some computations. In this case you already get the Py4J exception on the next operation that triggers communication with JVM process. You already knows about the problem and you're going to fix anyway. So additional reminder on `sc.stop()` was looked redundant for me.
    
    > what's the downside to giving this information versus making it impossible to tell that the JVM failed to shut down?
    
    Perfect question! Well, I don't have any strong argument here. It seems like a matter of taste about logging information usefulness.
    
    Ok, I'll add a warning. Does the following looks good for you?:
    ```python
    warning.warn(RuntimeWarning, 
                 'Unable to stop JVM process. Probably, it was crashed or externally killed')
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    **[Test build #3442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3442/consoleFull)** for PR 15961 at commit [`b47f12d`](https://github.com/apache/spark/commit/b47f12d45fe6d3ea6ecf45222e5903cb6c188a32).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by holdenk <gi...@git.apache.org>.
Github user holdenk commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    Great, can you maybe rebase one more time? For some reason in the PR dashboard its showing up as not mergeable even though github thinks it is.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by kxepal <gi...@git.apache.org>.
Github user kxepal commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    Hooray! \U0001f389 Thank you all for help here! (:


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15961: [SPARK-18523][PySpark]Make SparkContext.stop more reliab...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/15961
  
    **[Test build #3441 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3441/consoleFull)** for PR 15961 at commit [`518e31b`](https://github.com/apache/spark/commit/518e31b9a62faf7affc759ddf8df4e87500d6dc4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org