You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/26 00:19:35 UTC
[GitHub] [airflow] uranusjr opened a new pull request #15024: Action job for third-party inventory mirroring
uranusjr opened a new pull request #15024:
URL: https://github.com/apache/airflow/pull/15024
First part of #14989. This implements a job invoked in scheduled CI runs to fetch third-party Intersphinx inventories, and sync them to S3.
I’m currently putting the files in `s3://apache-airflow-docs/_inventory`.
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #15024: Action job for third-party inventory mirroring
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #15024:
URL: https://github.com/apache/airflow/pull/15024#discussion_r602551581
##########
File path: .github/workflows/ci.yml
##########
@@ -1250,3 +1250,28 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
tags: true
force: true
branch: master
+
+ intersphinx-inventory-mirror:
+ timeout-minutes: 10
+ name: "Update Intersphinx inventory mirror"
+ runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
+ if: github.event_name == 'schedule' && github.repository == 'apache/airflow'
+ needs: [ci-images]
+ steps:
+ - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
+ uses: actions/checkout@v2
+ with:
+ persist-credentials: false
+ submodules: recursive
+ - name: "Prepare CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
+ - name: "Fetch third-party module inventories"
+ run: ./scripts/ci/docs/ci_inventories.sh
+ - name: Configure AWS credentials
+ uses: ./.github/actions/configure-aws-credentials
+ with:
+ aws-access-key-id: ${{ secrets.DOCS_AWS_ACCESS_KEY_ID }}
+ aws-secret-access-key: ${{ secrets.DOCS_AWS_SECRET_ACCESS_KEY }}
+ aws-region: eu-central-1
+ - name: "Upload inventories to AWS S3"
+ run: aws s3 sync --delete ./files/inventory s3://apache-airflow-docs/_inventory
Review comment:
As discussed in #14989, github cache might be a much simpler solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #15024: Action job for third-party inventory mirroring
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #15024:
URL: https://github.com/apache/airflow/pull/15024#discussion_r602191124
##########
File path: .github/workflows/ci.yml
##########
@@ -1250,3 +1250,28 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
tags: true
force: true
branch: master
+
+ intersphinx-inventory-mirror:
+ timeout-minutes: 10
+ name: "Update Intersphinx inventory mirror"
+ runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
+ if: github.event_name == 'schedule' && github.repository == 'apache/airflow'
+ needs: [ci-images]
+ steps:
+ - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
+ uses: actions/checkout@v2
+ with:
+ persist-credentials: false
+ submodules: recursive
+ - name: "Prepare CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
+ - name: "Fetch third-party module inventories"
+ run: ./scripts/ci/docs/ci_inventories.sh
+ - name: Configure AWS credentials
+ uses: ./.github/actions/configure-aws-credentials
+ with:
+ aws-access-key-id: ${{ secrets.DOCS_AWS_ACCESS_KEY_ID }}
+ aws-secret-access-key: ${{ secrets.DOCS_AWS_SECRET_ACCESS_KEY }}
+ aws-region: eu-central-1
+ - name: "Upload inventories to AWS S3"
+ run: aws s3 sync --delete ./files/inventory s3://apache-airflow-docs/_inventory
Review comment:
This might be worth doing as a whole separate workflow. WDYT @potiuk?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed pull request #15024: Action job for third-party inventory mirroring
Posted by GitBox <gi...@apache.org>.
uranusjr closed pull request #15024:
URL: https://github.com/apache/airflow/pull/15024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #15024: Action job for third-party inventory mirroring
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #15024:
URL: https://github.com/apache/airflow/pull/15024#discussion_r602195062
##########
File path: .github/workflows/ci.yml
##########
@@ -1250,3 +1250,28 @@ ${{ hashFiles('.pre-commit-config.yaml') }}"
tags: true
force: true
branch: master
+
+ intersphinx-inventory-mirror:
+ timeout-minutes: 10
+ name: "Update Intersphinx inventory mirror"
+ runs-on: ${{ fromJson(needs.build-info.outputs.runsOn) }}
+ if: github.event_name == 'schedule' && github.repository == 'apache/airflow'
+ needs: [ci-images]
+ steps:
+ - name: "Checkout ${{ github.ref }} ( ${{ github.sha }} )"
+ uses: actions/checkout@v2
+ with:
+ persist-credentials: false
+ submodules: recursive
+ - name: "Prepare CI image ${{env.PYTHON_MAJOR_MINOR_VERSION}}:${{ env.GITHUB_REGISTRY_PULL_IMAGE_TAG }}"
+ run: ./scripts/ci/images/ci_prepare_ci_image_on_ci.sh
+ - name: "Fetch third-party module inventories"
+ run: ./scripts/ci/docs/ci_inventories.sh
+ - name: Configure AWS credentials
+ uses: ./.github/actions/configure-aws-credentials
+ with:
+ aws-access-key-id: ${{ secrets.DOCS_AWS_ACCESS_KEY_ID }}
+ aws-secret-access-key: ${{ secrets.DOCS_AWS_SECRET_ACCESS_KEY }}
+ aws-region: eu-central-1
+ - name: "Upload inventories to AWS S3"
+ run: aws s3 sync --delete ./files/inventory s3://apache-airflow-docs/_inventory
Review comment:
This reminds me, I intended to set `continue-on-error: true` but forgot. The sync job should not fail the entire action here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org