You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "Ma77Ball (via GitHub)" <gi...@apache.org> on 2026/06/01 20:12:39 UTC
[PR] ci: auto-sync docs/ to the website on doc changes [texera]
Ma77Ball opened a new pull request, #5308:
URL: https://github.com/apache/texera/pull/5308
### What changes were proposed in this PR?
- Adds a GitHub Actions workflow that keeps the website docs in sync with this repository, which is the single source of truth for documentation.
- How it works: when any file under `docs/` changes on `main`, the workflow copies each Markdown page body into the website repo (apache/incubator-texera-site) and pushes it, which triggers the site existing publish build.
- It preserves the website own front matter (page redirects and menu settings) and only replaces the body, so site navigation and aliases are never clobbered; new pages are copied and removed pages are deleted, so the site stays a faithful mirror.
- The job runs only on `apache/texera` (forks are skipped) and can also be triggered manually from the Actions tab.
### Any related issues, documentation, or discussions?
Related to: #5001
### How was this PR tested?
- Dry-ran the sync logic against the live website docs tree: the result is idempotent (a second run reports zero changes).
- Verified the website front matter (aliases, menu, linkTitle) is preserved while page bodies update.
- Validated the workflow YAML and the embedded Python both parse and compile cleanly.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3347182016
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
Review Comment:
Fixed. The loop now walks every file under `docs/`: Markdown pages get the front-matter merge, and other assets (such as the screenshot png) are copied verbatim. The delete pass also covers all file types, so removed assets are pruned too, keeping the site a faithful mirror.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
+
+ existed = tfile.exists()
+ if not existed or tfile.read_text(encoding="utf-8") != new_text:
+ tfile.parent.mkdir(parents=True, exist_ok=True)
+ tfile.write_text(new_text, encoding="utf-8")
+ if existed:
+ updated += 1
+ print(f" update {rel}")
+ else:
+ created += 1
+ print(f" create {rel}")
+
+ # Delete target pages that no longer exist in the source.
+ for tfile in sorted(target.rglob("*.md")):
+ rel = tfile.relative_to(target)
+ if rel not in source_rels:
+ tfile.unlink()
+ deleted += 1
+ print(f" delete {rel}")
+
+ print(f"Sync complete: {created} created, {updated} updated, {deleted} deleted.")
+ PY
+
+ - name: Commit and push to website
+ working-directory: site
+ env:
+ SOURCE_SHA: ${{ github.sha }}
+ SOURCE_REPO: ${{ github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ run: |
+ set -euo pipefail
+
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+
+ # Stop if the sync produced no changes.
+ git add -A content/docs/latest
+ if git diff --cached --quiet; then
+ echo "No documentation changes to sync."
+ exit 0
+ fi
+
+ short_sha="${SOURCE_SHA::7}"
+ git commit \
+ -m "docs: sync from ${SOURCE_REPO}@${short_sha}" \
+ -m "Automated sync of docs/ -> content/docs/latest/ from ${SOURCE_REPO}." \
+ -m "Source commit: ${SOURCE_SHA}" \
+ -m "Workflow run: ${RUN_URL}"
+
+ # Push, retrying with a rebase if main moved underneath us.
+ attempts=5
+ backoffs=(0 5 15 30 60)
+ for i in $(seq 0 $((attempts - 1))); do
+ if [[ "${backoffs[i]}" -gt 0 ]]; then
+ echo "Push attempt $((i + 1))/${attempts}: sleeping ${backoffs[i]}s"
+ sleep "${backoffs[i]}"
+ fi
+ if git push origin HEAD:main 2>&1; then
+ echo "Pushed synced docs to incubator-texera-site main."
+ exit 0
+ fi
+ echo "Push failed; refreshing origin/main and rebasing before retry."
+ git fetch --no-tags origin main
+ git rebase origin/main
Review Comment:
Fixed. The retry loop now checks the rebase result: on failure it runs `git rebase --abort`, emits a `::error::` annotation, and exits 1 instead of letting `set -e` kill the script mid-rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3347179165
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
Review Comment:
Good catch. `split_front_matter` now uses `splitlines()` and strips the fence lines, so it tolerates CRLF/CR endings and trailing whitespace on the `---` fences instead of treating front matter as body.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
Review Comment:
Checked the live site: its pages already carry exactly one blank line after the front matter, and `target_fm` ends with a single newline, so `target_fm + "\n" + body` reproduces that format. When the body is unchanged the regenerated text is byte-identical, so no diff is produced. Keeping it to match the site convention.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
Review Comment:
Fixed. The loop now walks every file under `docs/`: Markdown pages get the front-matter merge, and other assets (such as the screenshot png) are copied verbatim. The delete pass also covers all file types, so removed assets are pruned too, keeping the site a faithful mirror.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
+
+ existed = tfile.exists()
+ if not existed or tfile.read_text(encoding="utf-8") != new_text:
+ tfile.parent.mkdir(parents=True, exist_ok=True)
+ tfile.write_text(new_text, encoding="utf-8")
+ if existed:
+ updated += 1
+ print(f" update {rel}")
+ else:
+ created += 1
+ print(f" create {rel}")
+
+ # Delete target pages that no longer exist in the source.
+ for tfile in sorted(target.rglob("*.md")):
+ rel = tfile.relative_to(target)
+ if rel not in source_rels:
+ tfile.unlink()
+ deleted += 1
+ print(f" delete {rel}")
+
+ print(f"Sync complete: {created} created, {updated} updated, {deleted} deleted.")
+ PY
+
+ - name: Commit and push to website
+ working-directory: site
+ env:
+ SOURCE_SHA: ${{ github.sha }}
+ SOURCE_REPO: ${{ github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ run: |
+ set -euo pipefail
+
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+
+ # Stop if the sync produced no changes.
+ git add -A content/docs/latest
+ if git diff --cached --quiet; then
+ echo "No documentation changes to sync."
+ exit 0
+ fi
+
+ short_sha="${SOURCE_SHA::7}"
+ git commit \
+ -m "docs: sync from ${SOURCE_REPO}@${short_sha}" \
+ -m "Automated sync of docs/ -> content/docs/latest/ from ${SOURCE_REPO}." \
+ -m "Source commit: ${SOURCE_SHA}" \
+ -m "Workflow run: ${RUN_URL}"
+
+ # Push, retrying with a rebase if main moved underneath us.
+ attempts=5
+ backoffs=(0 5 15 30 60)
+ for i in $(seq 0 $((attempts - 1))); do
+ if [[ "${backoffs[i]}" -gt 0 ]]; then
+ echo "Push attempt $((i + 1))/${attempts}: sleeping ${backoffs[i]}s"
+ sleep "${backoffs[i]}"
+ fi
+ if git push origin HEAD:main 2>&1; then
+ echo "Pushed synced docs to incubator-texera-site main."
+ exit 0
+ fi
+ echo "Push failed; refreshing origin/main and rebasing before retry."
+ git fetch --no-tags origin main
+ git rebase origin/main
Review Comment:
Fixed. The retry loop now checks the rebase result: on failure it runs `git rebase --abort`, emits a `::error::` annotation, and exits 1 instead of letting `set -e` kill the script mid-rebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3347179165
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
Review Comment:
Now uses `splitlines()` + `strip()` on the fences, so CRLF/CR and trailing whitespace are handled.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
Review Comment:
`target_fm` ends in one `\n` and the site already has one blank line after front matter, so output is byte-identical and a no-op when the body is unchanged. Keeping as is.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
Review Comment:
Loop now walks all files: `.md` merges front matter, other assets are copied verbatim. Delete pass covers all file types too.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
+
+ existed = tfile.exists()
+ if not existed or tfile.read_text(encoding="utf-8") != new_text:
+ tfile.parent.mkdir(parents=True, exist_ok=True)
+ tfile.write_text(new_text, encoding="utf-8")
+ if existed:
+ updated += 1
+ print(f" update {rel}")
+ else:
+ created += 1
+ print(f" create {rel}")
+
+ # Delete target pages that no longer exist in the source.
+ for tfile in sorted(target.rglob("*.md")):
+ rel = tfile.relative_to(target)
+ if rel not in source_rels:
+ tfile.unlink()
+ deleted += 1
+ print(f" delete {rel}")
+
+ print(f"Sync complete: {created} created, {updated} updated, {deleted} deleted.")
+ PY
+
+ - name: Commit and push to website
+ working-directory: site
+ env:
+ SOURCE_SHA: ${{ github.sha }}
+ SOURCE_REPO: ${{ github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ run: |
+ set -euo pipefail
+
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+
+ # Stop if the sync produced no changes.
+ git add -A content/docs/latest
+ if git diff --cached --quiet; then
+ echo "No documentation changes to sync."
+ exit 0
+ fi
+
+ short_sha="${SOURCE_SHA::7}"
+ git commit \
+ -m "docs: sync from ${SOURCE_REPO}@${short_sha}" \
+ -m "Automated sync of docs/ -> content/docs/latest/ from ${SOURCE_REPO}." \
+ -m "Source commit: ${SOURCE_SHA}" \
+ -m "Workflow run: ${RUN_URL}"
+
+ # Push, retrying with a rebase if main moved underneath us.
+ attempts=5
+ backoffs=(0 5 15 30 60)
+ for i in $(seq 0 $((attempts - 1))); do
+ if [[ "${backoffs[i]}" -gt 0 ]]; then
+ echo "Push attempt $((i + 1))/${attempts}: sleeping ${backoffs[i]}s"
+ sleep "${backoffs[i]}"
+ fi
+ if git push origin HEAD:main 2>&1; then
+ echo "Pushed synced docs to incubator-texera-site main."
+ exit 0
+ fi
+ echo "Push failed; refreshing origin/main and rebasing before retry."
+ git fetch --no-tags origin main
+ git rebase origin/main
Review Comment:
On rebase failure the loop now runs `git rebase --abort`, emits `::error::`, and exits 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3346625969
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
Review Comment:
The front-matter splitter only detects YAML front matter when the file starts with "---\n" and when the closing fence line is exactly "---". This will fail on CRLF files ("---\r\n") and on common variants like trailing whitespace on the fence, causing the workflow to treat front matter as body and overwrite content unexpectedly.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
Review Comment:
`new_text` always inserts an extra blank line between the preserved front matter and the synced body (`target_fm` already ends with a newline). This changes file formatting unnecessarily and can cause noisy diffs/updates even when the body is identical.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
+
+ existed = tfile.exists()
+ if not existed or tfile.read_text(encoding="utf-8") != new_text:
+ tfile.parent.mkdir(parents=True, exist_ok=True)
+ tfile.write_text(new_text, encoding="utf-8")
+ if existed:
+ updated += 1
+ print(f" update {rel}")
+ else:
+ created += 1
+ print(f" create {rel}")
+
+ # Delete target pages that no longer exist in the source.
+ for tfile in sorted(target.rglob("*.md")):
+ rel = tfile.relative_to(target)
+ if rel not in source_rels:
+ tfile.unlink()
+ deleted += 1
+ print(f" delete {rel}")
+
+ print(f"Sync complete: {created} created, {updated} updated, {deleted} deleted.")
+ PY
+
+ - name: Commit and push to website
+ working-directory: site
+ env:
+ SOURCE_SHA: ${{ github.sha }}
+ SOURCE_REPO: ${{ github.repository }}
+ RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+ run: |
+ set -euo pipefail
+
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+
+ # Stop if the sync produced no changes.
+ git add -A content/docs/latest
+ if git diff --cached --quiet; then
+ echo "No documentation changes to sync."
+ exit 0
+ fi
+
+ short_sha="${SOURCE_SHA::7}"
+ git commit \
+ -m "docs: sync from ${SOURCE_REPO}@${short_sha}" \
+ -m "Automated sync of docs/ -> content/docs/latest/ from ${SOURCE_REPO}." \
+ -m "Source commit: ${SOURCE_SHA}" \
+ -m "Workflow run: ${RUN_URL}"
+
+ # Push, retrying with a rebase if main moved underneath us.
+ attempts=5
+ backoffs=(0 5 15 30 60)
+ for i in $(seq 0 $((attempts - 1))); do
+ if [[ "${backoffs[i]}" -gt 0 ]]; then
+ echo "Push attempt $((i + 1))/${attempts}: sleeping ${backoffs[i]}s"
+ sleep "${backoffs[i]}"
+ fi
+ if git push origin HEAD:main 2>&1; then
+ echo "Pushed synced docs to incubator-texera-site main."
+ exit 0
+ fi
+ echo "Push failed; refreshing origin/main and rebasing before retry."
+ git fetch --no-tags origin main
+ git rebase origin/main
Review Comment:
If `git rebase origin/main` fails (e.g., because the website repo changed the same docs files), `set -e` will terminate the script immediately without a clear error and without aborting the rebase state. Handling rebase failure explicitly makes the workflow behavior more reliable and easier to debug.
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
Review Comment:
The sync loop only processes `*.md` files, but `docs/` already contains non-Markdown assets (e.g., `docs/system-screenshot.png`). If an asset is added/changed/removed under `docs/`, this workflow will run (paths trigger is `docs/**`) but the asset will not be copied/deleted in the website repo, leaving the site out of sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on PR #5308:
URL: https://github.com/apache/texera/pull/5308#issuecomment-4596464879
/request-review @aglinxinyuan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3347179165
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
Review Comment:
Good catch. split_front_matter now uses splitlines() and strips the fence lines, so it tolerates CRLF/CR endings and trailing whitespace on the `---` fences instead of overwriting the body.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5308:
URL: https://github.com/apache/texera/pull/5308#issuecomment-4596125598
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/5308?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 49.97%. Comparing base ([`eb287f3`](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`685a76f`](https://app.codecov.io/gh/apache/texera/commit/685a76fc8f2bdee4ec27c11dcb56506432cef619?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #5308 +/- ##
=========================================
Coverage 49.97% 49.97%
Complexity 2421 2421
=========================================
Files 1052 1052
Lines 40780 40780
Branches 4356 4356
=========================================
Hits 20378 20378
Misses 19193 19193
Partials 1209 1209
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `41.89% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `33.76% <ø> (ø)` | | |
| [amber](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.01% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `1.38% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `54.68% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.42% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.43% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.80% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [python](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.80% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/5308/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `58.39% <ø> (ø)` | | Carriedforward from [eb287f3](https://app.codecov.io/gh/apache/texera/commit/eb287f3fc2cbba5416d851078affc10537123e68?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/texera/pull/5308?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on code in PR #5308:
URL: https://github.com/apache/texera/pull/5308#discussion_r3347180622
##########
.github/workflows/sync-docs-to-site.yml:
##########
@@ -0,0 +1,184 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Syncs docs/ into the website's content/docs/latest/ and pushes to the website.
+# Needs secret SITE_SYNC_TOKEN: a token with Contents:write on
+# apache/incubator-texera-site.
+
+name: Sync docs to website
+
+on:
+ push:
+ branches:
+ - main
+ paths:
+ - 'docs/**'
+ workflow_dispatch:
+
+# Run one sync at a time.
+concurrency:
+ group: sync-docs-to-site
+ cancel-in-progress: false
+
+permissions:
+ contents: read
+
+jobs:
+ sync:
+ # Skip on forks.
+ if: github.repository == 'apache/texera'
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout texera
+ uses: actions/checkout@v5
+ with:
+ path: texera
+
+ - name: Checkout incubator-texera-site
+ uses: actions/checkout@v5
+ with:
+ repository: apache/incubator-texera-site
+ ref: main
+ path: site
+ fetch-depth: 0
+ token: ${{ secrets.SITE_SYNC_TOKEN }}
+
+ - name: Sync docs/ into content/docs/latest/
+ env:
+ SOURCE_DOCS: texera/docs
+ TARGET_DOCS: site/content/docs/latest
+ run: |
+ python3 - <<'PY'
+ import os
+ import pathlib
+ import sys
+
+ source = pathlib.Path(os.environ["SOURCE_DOCS"])
+ target = pathlib.Path(os.environ["TARGET_DOCS"])
+
+
+ def split_front_matter(text):
+ # Split a page into (front matter, body) on the '---' fences.
+ if not text.startswith("---\n"):
+ return "", text
+ lines = text.split("\n")
+ for i in range(1, len(lines)):
+ if lines[i] == "---":
+ return "\n".join(lines[: i + 1]) + "\n", "\n".join(lines[i + 1 :])
+ return "", text
+
+
+ def normalize_body(body):
+ # Trim surrounding blank lines; "" if the body is empty.
+ body = body.lstrip("\n").rstrip()
+ return body + "\n" if body else ""
+
+
+ if not source.is_dir():
+ print(f"error: source dir not found: {source}", file=sys.stderr)
+ sys.exit(2)
+ target.mkdir(parents=True, exist_ok=True)
+
+ source_rels = set()
+ created = updated = deleted = 0
+
+ # For each source page: keep the target's front matter, use the source body.
+ for sfile in sorted(source.rglob("*.md")):
+ rel = sfile.relative_to(source)
+ source_rels.add(rel)
+ tfile = target / rel
+
+ src_text = sfile.read_text(encoding="utf-8")
+ _, src_body = split_front_matter(src_text)
+
+ if tfile.exists():
+ target_fm, _ = split_front_matter(tfile.read_text(encoding="utf-8"))
+ else:
+ target_fm, _ = split_front_matter(src_text)
+
+ body = normalize_body(src_body)
+ if body:
+ new_text = target_fm + ("\n" if target_fm else "") + body
+ else:
+ new_text = target_fm
Review Comment:
Checked the live site: its pages already carry exactly one blank line after the front matter, and `target_fm` ends with a single newline, so `target_fm + "\n" + body` reproduces that format. When the body is unchanged the regenerated text is byte-identical, so no diff is produced. Keeping it to match the site convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ci: auto-sync docs/ to the website on doc changes [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #5308:
URL: https://github.com/apache/texera/pull/5308
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org