You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/02/24 23:52:26 UTC
[GitHub] [incubator-mxnet] waytrue17 opened a new pull request #19955: [website] Automate website artifacts uploading
waytrue17 opened a new pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955
## Description ##
Port #19244 as a part of the MXNet release process for 2.0+ versions.
## Checklist ##
### Essentials ###
- [ ] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
- [ ] Changes are complete (i.e. I finished coding on this PR)
- [ ] All changes have test coverage
- [ ] Code is well-documented
### Changes ###
- [ ] Feature1, tests, (and when applicable, API doc)
- [ ] Feature2, tests, (and when applicable, API doc)
## Comments ##
- If this change is a backward incompatible change, why must this change be made.
- Interesting edge cases to note here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181370
Jenkins CI successfully triggered : [centos-gpu, centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [centos-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-791109134
Jenkins CI successfully triggered : [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-794282957
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] Zha0q1 merged pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
Zha0q1 merged pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789406906
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790254809
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-791173168
Jenkins CI successfully triggered : [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790978832
Jenkins CI successfully triggered : [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-793341375
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-792961000
> Seems that website builds started failing due to permission issue:
> https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-19986/1/pipeline
That's due to an s3 object permission setup issue when I manually uploading the versions.zip for testing. It's been fixed. This PR shouldn't have the issue as we set `--acl public-read` when uploading the file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789238212
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-791858965
Seems that website builds started failing due to permission issue:
https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-19986/1/pipeline
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-793144533
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-794631069
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-796267083
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789291818
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-791578547
Jenkins CI successfully triggered : [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [centos-cpu, centos-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] marcoabreu commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
marcoabreu commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r584379542
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -1212,6 +1212,30 @@ build_docs_beta() {
popd
}
+push_docs() {
+ folder_name=$1
+ set -ex
+ pip3 install --user awscli
+ export PATH=~/.local/bin:$PATH
+ pushd docs/_build
+ tar -xzf full_website.tgz --strip-components 1
+ # check if folder_name already exists in versions
+ pushd versions
+ if [ -d "$folder_name" ]; then
+ echo "Folder $folder_name already exists in versions. Please double check the FOLDER_NAME variable in Jenkens pipeline"
+ exit 1
+ fi
+ mv master $folder_name
+ popd
+ # back up versions folder
+ wget https://mxnet-website-static-artifacts.s3.us-east-2.amazonaws.com/versions.zip -O versions-backup.zip
+ aws s3 cp versions-backup.zip s3://mxnet-website-static-artifacts --acl public-read
+
+ zip -r9 versions.zip versions/.
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts --acl public-read
Review comment:
How about just publishing them in the format "YEAR-MONTH-DAY-COMMITHASH.zip" with a policy to recycle them after 3 months without access?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r585806747
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -1212,6 +1212,30 @@ build_docs_beta() {
popd
}
+push_docs() {
+ folder_name=$1
+ set -ex
+ pip3 install --user awscli
+ export PATH=~/.local/bin:$PATH
+ pushd docs/_build
+ tar -xzf full_website.tgz --strip-components 1
+ # check if folder_name already exists in versions
+ pushd versions
+ if [ -d "$folder_name" ]; then
+ echo "Folder $folder_name already exists in versions. Please double check the FOLDER_NAME variable in Jenkens pipeline"
+ exit 1
+ fi
+ mv master $folder_name
+ popd
+ # back up versions folder
+ wget https://mxnet-website-static-artifacts.s3.us-east-2.amazonaws.com/versions.zip -O versions-backup.zip
+ aws s3 cp versions-backup.zip s3://mxnet-website-static-artifacts --acl public-read
+
+ zip -r9 versions.zip versions/.
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts --acl public-read
Review comment:
This pipeline is for MXNet releases and should only be triggered to update the release version of website artifact. I think backing up by versions should be enough in this case. I changed the code to archive each released versions so the new back-up won't override the previous ones. The back-ups will have name `versions_backup_upto_$(release_version).zip'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-792961473
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-785470698
Hey @waytrue17 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [edge, unix-cpu, centos-cpu, clang, website, windows-cpu, windows-gpu, miscellaneous, centos-gpu, unix-gpu, sanity]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r584378696
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -1212,6 +1212,30 @@ build_docs_beta() {
popd
}
+push_docs() {
+ folder_name=$1
+ set -ex
+ pip3 install --user awscli
+ export PATH=~/.local/bin:$PATH
+ pushd docs/_build
+ tar -xzf full_website.tgz --strip-components 1
+ # check if folder_name already exists in versions
+ pushd versions
+ if [ -d "$folder_name" ]; then
+ echo "Folder $folder_name already exists in versions. Please double check the FOLDER_NAME variable in Jenkens pipeline"
+ exit 1
+ fi
+ mv master $folder_name
+ popd
+ # back up versions folder
+ wget https://mxnet-website-static-artifacts.s3.us-east-2.amazonaws.com/versions.zip -O versions-backup.zip
+ aws s3 cp versions-backup.zip s3://mxnet-website-static-artifacts --acl public-read
+
+ zip -r9 versions.zip versions/.
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts --acl public-read
Review comment:
one version might not be enough given that the pipeline runs daily and issues can go unnoticed longer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r588695556
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -1212,6 +1212,30 @@ build_docs_beta() {
popd
}
+push_docs() {
+ folder_name=$1
+ set -ex
+ pip3 install --user awscli
+ export PATH=~/.local/bin:$PATH
+ pushd docs/_build
+ tar -xzf full_website.tgz --strip-components 1
+ # check if folder_name already exists in versions
+ pushd versions
+ if [ -d "$folder_name" ]; then
+ echo "Folder $folder_name already exists in versions. Please double check the FOLDER_NAME variable in Jenkens pipeline"
+ exit 1
+ fi
+ mv master $folder_name
+ popd
+ zip -r9 versions.zip versions/.
+ # Upload versions folder
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts --acl public-read
+ # Backup versions folder with the latest version name
+ backup_file="versions_backup_upto_$folder_name.zip"
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts/$backup_file --acl public-read
Review comment:
Copy between s3 locations will make it more efficient.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
szha commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r583295668
##########
File path: ci/jenkins/Jenkins_steps.groovy
##########
@@ -1168,6 +1192,28 @@ def docs_publish_beta() {
}]
}
+// This is for uploading website artifacts to S3 bucket
+// Assumes you have run docs_full_website function
+def docs_upload_s3() {
+ return ['Upload artifacts to s3 bucket': {
+ node(NODE_LINUX_CPU) {
+ ws('workspace/docs') {
+ timeout(time: max_time, unit: 'MINUTES') {
+ if(env.FOLDER_NAME) {
+ utils.unpack_and_init('full_website', 'docs/_build/full_website.tgz')
Review comment:
can we make sure we have backups and can roll back if there's any issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789316685
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-786136449
@ys2843 @leezu Can you review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790089950
@mxnet-bot run ci [centos-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790089950
@mxnet-bot run ci [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790089950
@mxnet-bot run ci [unix-cpu,]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-795898511
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r585806747
##########
File path: ci/docker/runtime_functions.sh
##########
@@ -1212,6 +1212,30 @@ build_docs_beta() {
popd
}
+push_docs() {
+ folder_name=$1
+ set -ex
+ pip3 install --user awscli
+ export PATH=~/.local/bin:$PATH
+ pushd docs/_build
+ tar -xzf full_website.tgz --strip-components 1
+ # check if folder_name already exists in versions
+ pushd versions
+ if [ -d "$folder_name" ]; then
+ echo "Folder $folder_name already exists in versions. Please double check the FOLDER_NAME variable in Jenkens pipeline"
+ exit 1
+ fi
+ mv master $folder_name
+ popd
+ # back up versions folder
+ wget https://mxnet-website-static-artifacts.s3.us-east-2.amazonaws.com/versions.zip -O versions-backup.zip
+ aws s3 cp versions-backup.zip s3://mxnet-website-static-artifacts --acl public-read
+
+ zip -r9 versions.zip versions/.
+ aws s3 cp versions.zip s3://mxnet-website-static-artifacts --acl public-read
Review comment:
This pipeline is for MXNet releases and should only be triggered to update the release version of website artifact. I think backing up by versions should be enough in this case. I changed the code to archive each released versions so the new back-up won't override the previous ones. The archived artifact will have name `versions_backup_upto_$(release_version).zip'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790832240
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790090001
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790089950
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-789181285
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 commented on a change in pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 commented on a change in pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#discussion_r583958193
##########
File path: ci/jenkins/Jenkins_steps.groovy
##########
@@ -1168,6 +1192,28 @@ def docs_publish_beta() {
}]
}
+// This is for uploading website artifacts to S3 bucket
+// Assumes you have run docs_full_website function
+def docs_upload_s3() {
+ return ['Upload artifacts to s3 bucket': {
+ node(NODE_LINUX_CPU) {
+ ws('workspace/docs') {
+ timeout(time: max_time, unit: 'MINUTES') {
+ if(env.FOLDER_NAME) {
+ utils.unpack_and_init('full_website', 'docs/_build/full_website.tgz')
Review comment:
Added a back-up mechanism @szha
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] waytrue17 edited a comment on pull request #19955: [website] Automate website artifacts uploading
Posted by GitBox <gi...@apache.org>.
waytrue17 edited a comment on pull request #19955:
URL: https://github.com/apache/incubator-mxnet/pull/19955#issuecomment-790089950
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org