You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by po...@apache.org on 2025/05/15 09:54:02 UTC
(airflow) branch improve-live-staging-approach-for-docs updated (79038c3873d -> a29c8227d66)
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch improve-live-staging-approach-for-docs
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 79038c3873d Improve flow for live/staging doc publishing
add 5376179c45a Update upgrading_to_airflow3.rst (#50632)
add 08cc57d5bba Fix OpenAPI schema for `get_log` API (#50547)
new a29c8227d66 Improve flow for live/staging doc publishing
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (79038c3873d)
\
N -- N -- N refs/heads/improve-live-staging-approach-for-docs (a29c8227d66)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/publish-docs-to-s3.yml | 4 ++--
.../docs/installation/upgrading_to_airflow3.rst | 2 +-
.../src/airflow/api_fastapi/common/headers.py | 27 ++++++++++++++++++++++
.../src/airflow/api_fastapi/common/types.py | 1 +
.../core_api/openapi/v1-rest-api-generated.yaml | 8 ++++---
.../api_fastapi/core_api/routes/public/log.py | 13 ++++++-----
.../src/airflow/ui/openapi-gen/queries/common.ts | 2 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 2 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 2 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 2 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 2 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 2 +-
.../api_fastapi/core_api/routes/public/test_log.py | 14 +++++------
13 files changed, 55 insertions(+), 26 deletions(-)
(airflow) 01/01: Improve flow for live/staging doc publishing
Posted by po...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch improve-live-staging-approach-for-docs
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit a29c8227d66532827a3e00a0d2eb2e057117be1b
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Wed May 14 22:17:06 2025 -0400
Improve flow for live/staging doc publishing
There are a couple of improvements for live/staging doc publishing.
1) we use auto/live/staging rather than buckets in the drop-downs
2) auto selection of live/staging depending on the tag used to
produce the documentation
3) we use env variables to pass inputs/outputs to scripts - which
is good security practice
4) watermark changes are applied to documentation published to
staging site
---
.github/workflows/publish-docs-to-s3.yml | 65 +++++++++++++++++++++++---------
1 file changed, 48 insertions(+), 17 deletions(-)
diff --git a/.github/workflows/publish-docs-to-s3.yml b/.github/workflows/publish-docs-to-s3.yml
index 748a466cf8c..c71528b47fb 100644
--- a/.github/workflows/publish-docs-to-s3.yml
+++ b/.github/workflows/publish-docs-to-s3.yml
@@ -24,6 +24,15 @@ on: # yamllint disable-line rule:truthy
description: "The branch or tag to checkout for the docs publishing"
required: true
type: string
+ destination:
+ description: "The destination location in S3"
+ required: false
+ default: auto
+ type: choice
+ options:
+ - auto
+ - live
+ - staging
include-docs:
description: |
Space separated list of docs to build.
@@ -38,14 +47,6 @@ on: # yamllint disable-line rule:truthy
required: false
default: "no-docs-excluded"
type: string
- destination-location:
- description: "The destination location in S3, default is live site"
- required: false
- default: "s3://live-docs-airflow-apache-org/docs"
- type: choice
- options:
- - s3://live-docs-airflow-apache-org/docs
- - s3://staging-docs-airflow-apache-org/docs
env:
AIRFLOW_ROOT_PATH: "/home/runner/work/temp-airflow-repo-reference" # checkout dir for referenced tag
@@ -62,9 +63,11 @@ jobs:
REF: ${{ inputs.ref }}
INCLUDE_DOCS: ${{ inputs.include-docs }}
EXCLUDE_DOCS: ${{ inputs.exclude-docs }}
- DESTINATION_LOCATION: ${{ inputs.destination-location }}
+ DESTINATION: ${{ inputs.destination }}
outputs:
include-docs: ${{ inputs.include-docs == 'all' && '' || inputs.include-docs }}
+ destination-location: ${{ steps.parameters.outputs.destination-location }}
+ destination: ${{ steps.parameters.outputs.destination }}
if: contains(fromJSON('[
"ashb",
"eladkal",
@@ -78,13 +81,29 @@ jobs:
steps:
- name: "Input parameters summary"
shell: bash
+ id: parameters
run: |
echo "Input parameters summary"
echo "========================="
echo "Ref: '${REF}'"
echo "Included docs : '${INCLUDE_DOCS}'"
echo "Exclude docs: '${EXCLUDE_DOCS}'"
- echo "Destination location: '${DESTINATION_LOCATION}'"
+ echo "Destination: '${DESTINATION}'"
+ if [[ "${DESTINATION}" == "auto" ]]; then
+ if [[ "${REF}" =~ ^.*[0-9]*\.[0-9]*\.[0-9]*$ ]]; then
+ echo "${REF} looks like final release, using live destination"
+ DESTINATION="live"
+ else
+ echo "${REF} does not looks like final release, using staging destination"
+ DESTINATION="staging"
+ fi
+ fi
+ echo "destination=${DESTINATION}" >> ${GITHUB_OUTPUT}
+ if [[ "${DESTINATION}" == "live" ]]; then
+ echo "destination-location=s3://live-docs-airflow-apache-org/docs/" >> ${GITHUB_OUTPUT}
+ else
+ echo "destination-location=s3://staging-docs-airflow-apache-org/docs/" >> ${GITHUB_OUTPUT}
+ fi
build-ci-images:
name: Build CI images
@@ -179,18 +198,18 @@ jobs:
uses: actions/checkout@v4
with:
persist-credentials: false
- - name: "Prepare breeze & CI image: 3.9"
- uses: ./.github/actions/prepare_breeze_and_image
- with:
- platform: "linux/amd64"
- python: 3.9
- use-uv: true
- name: "Checkout ${{ inputs.ref }}"
run: |
git clone https://github.com/apache/airflow.git "${AIRFLOW_ROOT_PATH}"
cd "${AIRFLOW_ROOT_PATH}" && git checkout ${REF}
env:
REF: ${{ inputs.ref }}
+ - name: "Prepare breeze & CI image: 3.9"
+ uses: ./.github/actions/prepare_breeze_and_image
+ with:
+ platform: "linux/amd64"
+ python: 3.9
+ use-uv: true
- name: "Download docs prepared as artifacts"
uses: actions/download-artifact@v4
with:
@@ -220,6 +239,18 @@ jobs:
run: breeze release-management add-back-references docker-stack
- name: "Generate back references for helm-chart"
run: breeze release-management add-back-references helm-chart
+ - name: "Update watermarks"
+ env:
+ SOURCE_DIR_PATH: "/mnt/airflow-site/docs-archive/"
+ # yamllint disable rule:line-length
+ run: |
+ curl -sSf -o add_watermark.py https://raw.githubusercontent.com/apache/airflow-site/refs/heads/main/.github/scripts/add_watermark.py
+ chmod a+x add_watermark.py
+ mkdir -p images
+ curl -sSf -o images/staging.png https://raw.githubusercontent.com/apache/airflow-site/refs/heads/main/.github/scripts/images/staging.png
+ uv run add_watermark.py --pattern 'main.min*css' --folder ${SOURCE_DIR_PATH} \
+ --image-directory images --url-prefix /docs/images
+ if: needs.build-info.outputs.destination == 'staging'
- name: Install AWS CLI v2
run: |
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
@@ -235,7 +266,7 @@ jobs:
aws-region: us-east-2
- name: "Syncing docs to S3"
env:
- DESTINATION_LOCATION: "${{ inputs.destination-location }}"
+ DESTINATION_LOCATION: "${{ needs.build-info.outputs.destination-location }}"
SOURCE_DIR_PATH: "/mnt/airflow-site/docs-archive/"
EXCLUDE_DOCS: "${{ inputs.exclude-docs }}"
run: |