You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2025/05/16 21:25:02 UTC
(airflow-site-archive) branch add-back-back-reference-calculation created (now a24132615f)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch add-back-back-reference-calculation
in repository https://gitbox.apache.org/repos/asf/airflow-site-archive.git
at a24132615f Add back "back-reference" calculation
This branch includes the following new commits:
new 883bf4faf4 Add back "back-reference" calculation
new a24132615f Add back "back-reference" calculation
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(airflow-site-archive) 02/02: Add back "back-reference" calculation
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch add-back-back-reference-calculation
in repository https://gitbox.apache.org/repos/asf/airflow-site-archive.git
commit a24132615f3bc564044124c7e4cb37381a2d98de
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri May 16 17:18:54 2025 -0400
Add back "back-reference" calculation
I've removed back-reference calculation from airflow "Push docs to S3"
workflow - because it is not really possible to add back-references
there - as they are added in the "other" versions of the distribution
not in the current one - so when we release latest "3.0.2", back
references should be added in all other versions. This - therefore
should be done whenever we perform s3-to-github push from main in
the `airflow-site-archive` - because this is the place where we update
github based on s3 content and where we locally have all past
versions of the packages we work on.
---
.github/workflows/s3-to-github.yml | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/.github/workflows/s3-to-github.yml b/.github/workflows/s3-to-github.yml
index db3c8f6017..704fa3c587 100644
--- a/.github/workflows/s3-to-github.yml
+++ b/.github/workflows/s3-to-github.yml
@@ -52,6 +52,8 @@ jobs:
DOCUMENT_PACKAGES: ${{ inputs.document-packages }}
COMMIT_CHANGES: ${{ inputs.commit-changes }}
PROCESSES: ${{ inputs.processes }}
+ AIRFLOW_SITE_DIRECTORY: /mnt/cloned-airflow-site-archive/docs-archive
+
steps:
- name: Summarize parameters
id: parameters
@@ -160,6 +162,33 @@ jobs:
--processes "${PROCESSES}"
working-directory: /mnt/cloned-airflow-site-archive
+ - uses: actions/checkout@v4
+ # Checkout airflow to run breeze
+ with:
+ repository: 'apache/airflow'
+ path: airflow
+ fetch-depth: 1
+
+ - name: "Install breeze"
+ run: uv tool install -e ./dev/breeze
+ working-directory: airflow
+
+ - name: "Generate back references for providers"
+ run: breeze release-management add-back-references all-providers
+ working-directory: airflow
+
+ - name: "Generate back references for apache-airflow"
+ run: breeze release-management add-back-references apache-airflow
+ working-directory: airflow
+
+ - name: "Generate back references for docker-stack"
+ run: breeze release-management add-back-references docker-stack
+ working-directory: airflow
+
+ - name: "Generate back references for helm-chart"
+ run: breeze release-management add-back-references helm-chart
+ working-directory: airflow
+
- name: Show files that will be committed
run : git add --dry-run .
working-directory: /mnt/cloned-airflow-site-archive
(airflow-site-archive) 01/02: Add back "back-reference" calculation
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch add-back-back-reference-calculation
in repository https://gitbox.apache.org/repos/asf/airflow-site-archive.git
commit 883bf4faf49e47441626e69ed2808fa0399abe30
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Fri May 16 17:08:51 2025 -0400
Add back "back-reference" calculation
I've removed back-reference calculation from airlfow "Push docs to S3"
workflow - because it is not really possible to add back-references
there - as they are added in the "other" versions of the distribution
not in the current one - so when we release latest "3.0.2", back
references should be added in all other versions. This - therefore
should be done whenever we peform s3-to-github push from main in
the `airflow-site-archive` - because this is the place where we update
github based on s3 content and where we locally have all past
versions of the packages we work on.
---
.github/workflows/github-to-s3.yml | 52 +++++++++++++++++++++++++++++++++++---
1 file changed, 49 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/github-to-s3.yml b/.github/workflows/github-to-s3.yml
index d38cb6d0ee..cff9086a8b 100644
--- a/.github/workflows/github-to-s3.yml
+++ b/.github/workflows/github-to-s3.yml
@@ -52,6 +52,8 @@ jobs:
github-to-s3:
name: GitHub to S3
runs-on: ubuntu-latest
+ env:
+ AIRFLOW_SITE_DIRECTORY: /mnt/cloned-airflow-site-archive/docs-archive
steps:
- name: Summarize parameters
id: parameters
@@ -97,6 +99,9 @@ jobs:
sudo /tmp/aws/install --update
rm -rf /tmp/aws/
+ - name: "Install uv"
+ run: curl -LsSf https://astral.sh/uv/install.sh | sh
+
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@010d0da01d0b5a38af31e9c3470dbfdabdecca3a # v4.0.1
with:
@@ -111,6 +116,13 @@ jobs:
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
+ - uses: actions/checkout@v4
+ # Checkout airflow to run breeze
+ with:
+ repository: 'apache/airflow'
+ path: airflow
+ fetch-depth: 1
+
- name: "Checkout ${{ github.ref }} ( ${{ github.sha }} ) for scripts"
uses: actions/checkout@v4
# Checkout only workflow and scripts directory to run scripts from
@@ -153,6 +165,7 @@ jobs:
echo "EOF" >> ${GITHUB_OUTPUT}
set -x
echo "docs-packages-processed=${packages}" >> ${GITHUB_OUTPUT}
+
- name: >
Checkout (${{ steps.parameters.outputs.commit-ref }}) to /mnt/cloned-airflow-site-archive
with docs packages: ${{ steps.docs-packages-processed.outputs.docs-packages-processed }}
@@ -164,6 +177,7 @@ jobs:
${{ steps.docs-packages-processed.outputs.sparse-checkout }}
ref: ${{ steps.parameters.outputs.commit-ref }}
if: steps.docs-packages-processed.outputs.docs-packages-processed != 'all'
+
- name: >
Checkout (${{ steps.parameters.outputs.commit-ref }}) to /mnt/cloned-airflow-site-archive (all packages)
uses: actions/checkout@v4
@@ -172,12 +186,44 @@ jobs:
fetch-depth: 2
ref: ${{ steps.parameters.outputs.commit-ref }}
if: steps.docs-packages-processed.outputs.docs-packages-processed == 'all'
- - name: "Install uv"
- run: curl -LsSf https://astral.sh/uv/install.sh | sh
+
+ - name: "Install breeze"
+ run: uv tool install -e ./dev/breeze
+ working-directory: airflow
+
+ - name: "Generate back references for providers"
+ run: breeze release-management add-back-references all-providers
+ working-directory: airflow
+
+ - name: "Generate back references for apache-airflow"
+ run: breeze release-management add-back-references apache-airflow
+ working-directory: airflow
+
+ - name: "Generate back references for docker-stack"
+ run: breeze release-management add-back-references docker-stack
+ working-directory: airflow
+
+ - name: "Generate back references for helm-chart"
+ run: breeze release-management add-back-references helm-chart
+ working-directory: airflow
+
+ - name: "Commit back references if they are added in main"
+ run: |
+ if [[ -n "$(git status -s)" ]]; then
+ git config --global user.name "GitHub Action"
+ git config --global user.email "
+ git add .
+ git commit -m "Add back references"
+ git push origin main
+ fi
+ working-directory: /mnt/cloned-airflow-site-archive
+ if : steps.docs-packages-processed.outputs.commit-ref == 'main'
+
- name: Add staging watermarks
# yamllint disable rule:line-length
run: |
- curl -sSf -o add_watermark.py https://raw.githubusercontent.com/apache/airflow-site/refs/heads/main/.github/scripts/add_watermark.py
+ curl -sSf -o add_watermark.py https://raw.githubusercontent.com/apache/airflow-site/refs/heads/main/.github/scripts/add_watermark.py \
+ --header "Authorization: Bearer ${{ github.token }} " --header "X-GitHub-Api-Version: 2022-11-28"
chmod a+x add_watermark.py
mkdir -p images
curl -sSf -o images/staging.png https://raw.githubusercontent.com/apache/airflow-site/refs/heads/main/.github/scripts/images/staging.png