You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by HyukjinKwon <gi...@git.apache.org> on 2018/08/14 02:03:11 UTC

[GitHub] spark pull request #22098: [SPARK-24886][INFRA] Fix the testing script to in...

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/22098

    [SPARK-24886][INFRA] Fix the testing script to increase timeout for Jenkins build (from 340m to 400m)

    ## What changes were proposed in this pull request?
    
    This PR targets to increase the timeout from 340 to 400m. Please also see https://github.com/apache/spark/pull/21845#discussion_r209807634
    
    ## How was this patch tested?
    
    N/A


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark SPARK-24886-1

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/22098.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22098
    
----
commit bffce5ef7861c03f9d70f7e8e193747d3b2cce2b
Author: hyukjinkwon <gu...@...>
Date:   2018-08-14T02:00:07Z

    Fix the testing script to increase timeout for Jenkins build (from 340m to 400m)

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by shaneknapp <gi...@git.apache.org>.

Github user shaneknapp commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    to be honest, i have *no* idea why this was done in the first place...  the
    jenkins build already has a timeout.  i'll dig around today and see if i
    can't find out who put this in the repo in the first place.
    
    
    
    On Tue, Aug 14, 2018 at 12:58 AM, Hyukjin Kwon <no...@github.com>
    wrote:
    
    > Hm.. makes sense. @shaneknapp <https://github.com/shaneknapp> WDYT? I can
    > also log the timeout as well and default to 400m if not set just in case
    > for now. Would this make the things complicated? If so, let me just leave
    > it as is.
    >
    > —
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub
    > <https://github.com/apache/spark/pull/22098#issuecomment-412787056>, or mute
    > the thread
    > <https://github.com/notifications/unsubscribe-auth/ABiDrLAUnGOCZM2w7r__4XIM4eBf7JxVks5uQoMNgaJpZM4V7rKC>
    > .
    >



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by shaneknapp <gi...@git.apache.org>.

Github user shaneknapp commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    >
    > Ah, @shaneknapp <https://github.com/shaneknapp>, probably one possibility
    > about the motivation is that we don't touch build configuration often and
    > control the build time within the codebase so that committers don't have to
    > bother you every time it needs. For instance, although I know how to
    > control the build time too, I don't touch them.
    >
    > that makes sense, but at the same time, it does require coordination to
    happen between the timeout in the test framework and setting the jenkins
    (absolute timeout)...  which literally is just adding another step to the
    process.
    
    > Shall we just keep the current way and discuss if we happen to increase it
    > again? I think 400m looks very enough as a limit, and we will put some
    > efforts to reduce or keep the current limit I believe.
    >
    >  sure, this works for me.  and seeing that our builds are now taking
    upwards of ***6*** hours, i'd rather have a conversation about build
    strategy before revisiting this.  :)
    
    >
    >



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    Ah, @shaneknapp, probably one possibility about the motivation is that we don't touch build configuration often and control the build time within the codebase so that committers don't have to bother you every time it needs. For instance, although I know how to control the build time too, I don't touch them.
    
    Shall we just keep the current way and discuss if we happen to increase it again? I think 400m looks very enough as a limit, and we will put some efforts to reduce or keep the current limit I believe.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    cc @shaneknapp 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    **[Test build #94722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94722/testReport)** for PR 22098 at commit [`bffce5e`](https://github.com/apache/spark/commit/bffce5ef7861c03f9d70f7e8e193747d3b2cce2b).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    **[Test build #94722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94722/testReport)** for PR 22098 at commit [`bffce5e`](https://github.com/apache/spark/commit/bffce5ef7861c03f9d70f7e8e193747d3b2cce2b).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    @shaneknapp, seems this was first introduced in https://issues.apache.org/jira/browse/SPARK-3076 / https://github.com/apache/spark/pull/1974 for a good reason fwiw.
    
    One thing I am not sure is if we need to have env or not .. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    Hm.. makes sense. @shaneknapp WDYT? I can also log the timeout as well and default to `400m` if not set just in case for now. Would this make the things complicated? If so, let me just leave it as is.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by shaneknapp <gi...@git.apache.org>.

Github user shaneknapp commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    oh wow, nice find @HyukjinKwon!  code archaeology ftw!  :)
    
    anyways:  that PR went through about a week after i started here @ the amplab and i don't think i even had access to the build system at that point.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    haha, it's more then 4 years ago .. if we are unsure on the env, let me just push this in.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2161/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by HyukjinKwon <gi...@git.apache.org>.

Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    Let me just push this in.
    
    Merged to master.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22098: [SPARK-24886][INFRA] Fix the testing script to increase ...

Posted by gengliangwang <gi...@git.apache.org>.

Github user gengliangwang commented on the issue:

    https://github.com/apache/spark/pull/22098
  
    One suggestion: can we add a environment variable for the script execution timeout in Jenkins, so that we don't need to modify the hardcode value here?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org