You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/04/08 23:07:46 UTC
[GitHub] [incubator-mxnet] josephevans opened a new pull request #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
josephevans opened a new pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999
## Description ##
This changes causes the large CI pipelines to wait for a sanity build to complete successfully for the PR before running. If the sanity build cannot be found, it continues on as currently.
https://github.com/apache/incubator-mxnet/issues/17802
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r409130525
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
+ build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+ ['centos-cpu', 'centos-gpu', 'clang', 'edge',
Review comment:
Okay agree.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406149155
##########
File path: ci/Jenkinsfile_utils.groovy
##########
@@ -261,6 +261,41 @@ def assign_node_labels(args) {
NODE_UTILITY = args.utility
}
+def sanity_build_running(pr) {
+ def retVal = "UNKNOWN"
+ try {
+ def prSanityBuild = Jenkins.instance.getItem("mxnet-validation").getItem("sanity").getItem(pr)
+ if (prSanityBuild.isBuilding() || prSanityBuild.isInQueue()) {
+ echo "Sanity build " + prSanityBuild.getLastBuild().number + " currently running"
+ retVal = "RUNNING"
+ } else {
+ retVal = prSanityBuild.getLastBuild().result.toString()
+ }
+ } catch (checkError) {
+ echo "Caught exception trying to find sanity build for this PR, continuing build without delay."
+ }
+ return retVal
+}
+
+def wait_for_sanity_builds() {
+ if (env.CHANGE_ID && env.JOB_NAME.lastIndexOf("PR-") != -1) {
+ echo "This is a pull request (PR-${env.CHANGE_ID}), checking for running sanity build."
+ // sleep to make sure sanity job is created
+ sleep(10)
+ def PRNum = env.JOB_NAME.substring(env.JOB_NAME.lastIndexOf("PR-"))
+ def buildStatus = sanity_build_running(PRNum)
+ while (buildStatus.equals("RUNNING")) {
+ sleep(20)
+ buildStatus = sanity_build_running(PRNum)
+ }
+ echo "Sanity build result: " + buildStatus
+ if (buildStatus.equals("FAILURE") || buildStatus.equals("ABORTED")) {
Review comment:
Check for success, not for failure. The default should be to stop if something unexpected happened, not to continue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-613157589
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406888453
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
+ build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+ ['centos-cpu', 'centos-gpu', 'clang', 'edge',
Review comment:
Is there some way to build this list automatically?
Have you tested this setup?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611244412
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r409081524
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
@marcoabreu Thanks for the suggestions, I've update the PR to preserve the full path of the build job, so the staggered build pipelines will still work if we reorganize our Jenkins pipelines into subfolders.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406148960
##########
File path: ci/Jenkinsfile_utils.groovy
##########
@@ -261,6 +261,41 @@ def assign_node_labels(args) {
NODE_UTILITY = args.utility
}
+def sanity_build_running(pr) {
+ def retVal = "UNKNOWN"
+ try {
+ def prSanityBuild = Jenkins.instance.getItem("mxnet-validation").getItem("sanity").getItem(pr)
+ if (prSanityBuild.isBuilding() || prSanityBuild.isInQueue()) {
+ echo "Sanity build " + prSanityBuild.getLastBuild().number + " currently running"
+ retVal = "RUNNING"
+ } else {
+ retVal = prSanityBuild.getLastBuild().result.toString()
+ }
+ } catch (checkError) {
+ echo "Caught exception trying to find sanity build for this PR, continuing build without delay."
+ }
+ return retVal
+}
+
+def wait_for_sanity_builds() {
+ if (env.CHANGE_ID && env.JOB_NAME.lastIndexOf("PR-") != -1) {
+ echo "This is a pull request (PR-${env.CHANGE_ID}), checking for running sanity build."
+ // sleep to make sure sanity job is created
+ sleep(10)
+ def PRNum = env.JOB_NAME.substring(env.JOB_NAME.lastIndexOf("PR-"))
+ def buildStatus = sanity_build_running(PRNum)
+ while (buildStatus.equals("RUNNING")) {
+ sleep(20)
Review comment:
Why so long?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408510926
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
+ build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+ ['centos-cpu', 'centos-gpu', 'clang', 'edge',
Review comment:
Which function exactly would have to be whitelisted? Maybe we can make an exception if it does not expose anything we're worried about.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611239172
Hey @josephevans , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [clang, windows-cpu, sanity, windows-gpu, website, miscellaneous, centos-cpu, unix-cpu, unix-gpu, centos-gpu, edge]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611248598
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406843879
##########
File path: ci/Jenkinsfile_utils.groovy
##########
@@ -261,6 +261,41 @@ def assign_node_labels(args) {
NODE_UTILITY = args.utility
}
+def sanity_build_running(pr) {
+ def retVal = "UNKNOWN"
+ try {
+ def prSanityBuild = Jenkins.instance.getItem("mxnet-validation").getItem("sanity").getItem(pr)
+ if (prSanityBuild.isBuilding() || prSanityBuild.isInQueue()) {
+ echo "Sanity build " + prSanityBuild.getLastBuild().number + " currently running"
+ retVal = "RUNNING"
+ } else {
+ retVal = prSanityBuild.getLastBuild().result.toString()
+ }
+ } catch (checkError) {
+ echo "Caught exception trying to find sanity build for this PR, continuing build without delay."
+ }
+ return retVal
+}
+
+def wait_for_sanity_builds() {
+ if (env.CHANGE_ID && env.JOB_NAME.lastIndexOf("PR-") != -1) {
+ echo "This is a pull request (PR-${env.CHANGE_ID}), checking for running sanity build."
+ // sleep to make sure sanity job is created
+ sleep(10)
Review comment:
Refactored to just use a new build pipeline. No sleep or having to check on other build pipelines.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611262534
> Looks like the other builds have already started running before sanity build completed @josephevans
Yeah, there's a few script calls I have to approve in Jenkins for this to work. That's why it skipped the sanity check delays initially (it's fail open.) It'd be nice to use a trusted library in Jenkins to do this to (a) protect against potential security vulnerabilities and malicious abuse and (b) not have to approve these in the future. Looking into that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611244381
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406888223
##########
File path: ci/Jenkinsfile_utils.groovy
##########
@@ -261,6 +261,41 @@ def assign_node_labels(args) {
NODE_UTILITY = args.utility
}
+def sanity_build_running(pr) {
+ def retVal = "UNKNOWN"
+ try {
+ def prSanityBuild = Jenkins.instance.getItem("mxnet-validation").getItem("sanity").getItem(pr)
+ if (prSanityBuild.isBuilding() || prSanityBuild.isInQueue()) {
+ echo "Sanity build " + prSanityBuild.getLastBuild().number + " currently running"
+ retVal = "RUNNING"
+ } else {
+ retVal = prSanityBuild.getLastBuild().result.toString()
+ }
+ } catch (checkError) {
+ echo "Caught exception trying to find sanity build for this PR, continuing build without delay."
+ }
+ return retVal
+}
+
+def wait_for_sanity_builds() {
+ if (env.CHANGE_ID && env.JOB_NAME.lastIndexOf("PR-") != -1) {
+ echo "This is a pull request (PR-${env.CHANGE_ID}), checking for running sanity build."
+ // sleep to make sure sanity job is created
+ sleep(10)
Review comment:
Awesome!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest merged pull request #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
apeforest merged pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408076406
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
Are you sure that this way is resilient? Jobs might have multiple slashes etc. This doesn't seem to be fully reliable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-613157558
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611335197
Makes sense. Thanks for the clarification
+1 for using secure functions instead of insecure.
I remember facing the issue with printStackTrace requiring Admin approval for insecure script (which led to Jenkins build status not being posted on Github for brief period).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-614698018
@mxnet-label-bot [pr-awaiting-merge]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408473758
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
+ build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+ ['centos-cpu', 'centos-gpu', 'clang', 'edge',
Review comment:
This has all been tested on mxnet-ci-dev.
We can get a list from Jenkins, but it would require scriptApprovals, which I'd like to avoid. I could move the build list to a variable at the top of the file so it's more clear.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-614770098
We will still need to make some changes to Jenkins in order for the staggered builds to start working, so there is no risk in merging now. I will send an email out to the community when I setup the new build pipeline and migrate to it.
Here is a test run we did on ci-dev. As you can see, we first run the sanity build. Once it finishes, we spawn off all the other build jobs to run in the background.
http://jenkins.mxnet-ci-dev.amazon-ml.com/job/mxnet-validation-joeev/job/full-build/view/change-requests/job/PR-2/16/consoleText
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408655335
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
It would be helpful to get somebody else involved who's familiar with Jenkins. I personally don't have enough time to do coaching
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-614739736
Please show that this is tested and working before merging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] sandeep-krishnamurthy commented on a change in
pull request #17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
sandeep-krishnamurthy commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408585601
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
Thanks for your contribution @josephevans and review @marcoabreu . How can I assist here? Looks like a regular dev discussion as with any other PRs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r409054233
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
+ build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+ ['centos-cpu', 'centos-gpu', 'clang', 'edge',
Review comment:
We would have to whitelist the Jenkins.instance static method, which would open a huge hole that anyone who submitted a PR could exploit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r406148873
##########
File path: ci/Jenkinsfile_utils.groovy
##########
@@ -261,6 +261,41 @@ def assign_node_labels(args) {
NODE_UTILITY = args.utility
}
+def sanity_build_running(pr) {
+ def retVal = "UNKNOWN"
+ try {
+ def prSanityBuild = Jenkins.instance.getItem("mxnet-validation").getItem("sanity").getItem(pr)
+ if (prSanityBuild.isBuilding() || prSanityBuild.isInQueue()) {
+ echo "Sanity build " + prSanityBuild.getLastBuild().number + " currently running"
+ retVal = "RUNNING"
+ } else {
+ retVal = prSanityBuild.getLastBuild().result.toString()
+ }
+ } catch (checkError) {
+ echo "Caught exception trying to find sanity build for this PR, continuing build without delay."
+ }
+ return retVal
+}
+
+def wait_for_sanity_builds() {
+ if (env.CHANGE_ID && env.JOB_NAME.lastIndexOf("PR-") != -1) {
+ echo "This is a pull request (PR-${env.CHANGE_ID}), checking for running sanity build."
+ // sleep to make sure sanity job is created
+ sleep(10)
Review comment:
That's a ballsy assumption buddy. Please find a way to reliably check the status - sleeps are not the way to go.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611241841
Looks like the other builds are running despite sanity build not completed @josephevans
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611248577
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408510404
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
Well but this will break with nested folders (a/b/c/sanity will result in a/sanity) or if there is no top-level folder at all. @szha @sandeep-krishnamurthy can you get somebody to assist on this matter?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] josephevans commented on a change in pull request
#17999: For mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
josephevans commented on a change in pull request #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#discussion_r408471610
##########
File path: ci/jenkins/Jenkinsfile_full
##########
@@ -0,0 +1,36 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+max_time = 180
+
+stage("full-build") {
+ def pipelineName = JOB_NAME.substring(0, JOB_NAME.indexOf('/'))
Review comment:
Yes, this will be resilient. It just extracts the parent folder of the job pipeline (ie 'mxnet-validation'.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya edited a comment on issue #17999: For
mxnet-validation pipeline,
require sanity build to complete successfully before running other build
pipelines.
Posted by GitBox <gi...@apache.org>.
ChaiBapchya edited a comment on issue #17999: For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines.
URL: https://github.com/apache/incubator-mxnet/pull/17999#issuecomment-611241841
Looks like the other builds have already started running before sanity build completed @josephevans
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services