You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by FRosner <gi...@git.apache.org> on 2017/10/17 12:01:51 UTC

[GitHub] spark pull request #19514: [SPARK-21551][Python] Increase timeout for Python...

GitHub user FRosner opened a pull request:

    https://github.com/apache/spark/pull/19514

    [SPARK-21551][Python] Increase timeout for PythonRDD.serveIterator

    Backport of https://github.com/apache/spark/pull/18752 (https://issues.apache.org/jira/browse/SPARK-21551)
    
    (cherry picked from commit 9d3c6640f56e3e4fd195d3ad8cead09df67a72c7)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/FRosner/spark branch-2.0

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19514.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19514
    
----
commit 7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7
Author: peay <pe...@protonmail.com>
Date:   2017-07-27T20:49:28Z

    [SPARK-21551][Python] Increase timeout for PythonRDD.serveIterator
    
    (cherry picked from commit 9d3c6640f56e3e4fd195d3ad8cead09df67a72c7)

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #82901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82901/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    I think we can safely merge this change as it clearly passes any tests whose functionality it would affect. We could defer further discussion about what to do about CRAN versions elsewhere, yes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    I haven't tried it, but it might sound like it will counter-act -as-cran check sets completely?
    ```
    R_CHECK_CRAN_INCOMING_
    Check whether package is suitable for publication on CRAN. Default: false, except for CRAN submission checks.
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    @srowen, not a big deal but .. I believe it's merged to 2.0 instead of master though.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    We didn't foresee this but it looks like `R CMD check --as-cran` throws this error if we try to build a package with a version number older than the one uploaded to CRAN.
    
    There are a couple of ways around this -- we can set an environment variable `_R_CHECK_CRAN_INCOMING_= false` (documented in [1]) or we can change our `check-cran.sh` to admit one more `WARNING`. This would of course only be done for `branch-2.0`
    
    Any thoughts @felixcheung @HyukjinKwon ?
    
    [1] https://cran.r-project.org/doc/manuals/r-release/R-ints.html


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #3956 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3956/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #3949 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3949/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #19514: [SPARK-21551][Python] Increase timeout for Python...

Posted by FRosner <gi...@git.apache.org>.
Github user FRosner closed the pull request at:

    https://github.com/apache/spark/pull/19514


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #82883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82883/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Merged to master @FRosner ; you'll have to close this manually too


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #3956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3956/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).
     * This patch **fails SparkR unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #82901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82901/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).
     * This patch **fails SparkR unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82901/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    @shivaram do you know if this error means branch 2.0 won't pass R CRAN checks at this point?
    
    ```
    Running CRAN check with --as-cran --no-tests --no-manual options
    * using log directory '/home/jenkins/workspace/SparkPullRequestBuilder/R/SparkR.Rcheck'
    * using R version 3.1.1 (2014-07-10)
    * using platform: x86_64-redhat-linux-gnu (64-bit)
    * using session charset: ASCII
    * using option '--no-tests'
    * checking for file 'SparkR/DESCRIPTION' ... OK
    * checking extension type ... Package
    * this is package 'SparkR' version '2.0.3'
    * checking CRAN incoming feasibility ... WARNING
    Maintainer: 'Shivaram Venkataraman <sh...@cs.berkeley.edu>'
    Insufficient package version (submitted: 2.0.3, existing: 2.1.2)
    Days since last update: 6
    ```


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Thanks, I have been following it @shivaram and @felixcheung. Separate JIRA sounds good to me and I am okay witn merging it.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    whoa.
    `_R_CHECK_CRAN_INCOMING_= false` sounds like the right approach. I'm a bit concerned with blindly letting through one more warning though, perhaps grep for the specific warning text and only let one more through if it matches?



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Good point. I'm not sure it counteracts it completely. We should run it to see the behavior I guess.
    
    I am not a big fan of mucking with Jenkins versions because it fundamentally looks like CRAN doesn't like us pushing newer versions from older branches ? For example if we release 2.2.1 then we can't submit 2.1.3 to CRAN. We should first discuss if we are okay with that -- we can move this to a JIRA ?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Test FAILed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82883/
    Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Can one of the admins verify this patch?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    **[Test build #82883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82883/consoleFull)** for PR 19514 at commit [`7ec2dc7`](https://github.com/apache/spark/commit/7ec2dc70ce26ff754c0ea38f3cf9964d67bc62f7).
     * This patch **fails SparkR unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    LGTM too BTW.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Merged build finished. Test FAILed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    retest this please


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/19514
  
    Oops yes meant 2.0


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org