You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by "nchammas (via GitHub)" <gi...@apache.org> on 2026/07/01 14:49:23 UTC
[PR] Add workflow to automatically generate and commit site HTML [spark-website]
nchammas opened a new pull request, #697:
URL: https://github.com/apache/spark-website/pull/697
[As proposed here][1], I believe it would be best to overhaul the workflow for this project by splitting the source into a `master` branch, and having `asf-site` be just for the generated HTML. We could even have per-branch staging sites generated and published automatically for us by ASF.
However, this PR takes a more conservative approach as suggested by @cloud-fan and adds a new workflow that generates the docs and pushes the HTML automatically to `asf-site`.
TODO:
- [ ] Update the README with simpler contribution instructions.
- [ ] Figure out if there is a way to ignore `site/` locally since it's being generated and committed by the workflow now.
[1]: https://github.com/apache/spark-website/pull/696#issuecomment-4773921749
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4947456371
yea there is no staging site AFAIK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4947656798
I think the problem is `content/`. I will open a new PR after we first get the site back up with all the same changes here, minus the deletion of the `content/` symlink. I will also add some documentation for it so that nobody else assumes it is no longer needed. 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4857989564
Thank you to whoever approved the previous workflows. I need another approval.
Could I perhaps be added to some workflow whitelist so maintainers don't have to manually approve my workflows every time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4920580139
> as long as it does not include this [html] marker
Can we use a more internal marker that no one will accidentally use in their own PR title?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan merged PR #697:
URL: https://github.com/apache/spark-website/pull/697
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4947531567
Looks like something went wrong! Perhaps `content/` is needed after all? I'm not sure. This is where a staging site would help.
Better revert for now and I'll look into this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4876874724
> Could I perhaps be added to some workflow whitelist so maintainers don't have to manually approve my workflows every time?
@cloud-fan - Checking in again about this. Not absolutely required, obviously, but it would make it easier for me to contribute here.
I understand there is a repo setting to require workflow approval only for first-time contributors to a repo. You can also add me specifically as a read-only repo collaborator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4856963065
I need a maintainer to approve the workflow runs so I can test my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4926769812
I kept the `[html]` marker but updated the condition to also check the commit author: https://github.com/apache/spark-website/blob/bf80b5efe29c9e26d0a3f66058ebe43fe325c419/.github/workflows/html-push.yml#L15-L19
That way, even if contributors accidentally use `[html]` in their commit message, it won't have any impact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4916969533
to be clear, the post-merge job will generate HTML for the current codebase, so even if something goes wrong, a later post-merge job will make md files and HTML fully in sync again?
BTW I don't think Apache allow us to add a workflow approver, so it has to be approved every single time...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4917280585
> to be clear, the post-merge job will generate HTML for the current codebase, so even if something goes wrong, a later post-merge job will make md files and HTML fully in sync again?
Correct.
Every commit to `asf-site` will trigger a build of the HTML. A new commit will be pushed with `[html] ` prepended to the title of the latest commit on the branch. The `[html]` string in the title is used as a marker to prevent the workflow from building and pushing the HTML again.
https://github.com/apache/spark-website/blob/6eab236e822a55edc2bf3f834bf359cb6c2bca0c/.github/workflows/html-push.yml#L13-L15
Whenever a new commit is pushed to `asf-site`, as long as it does not include this `[html]` marker, that will trigger a new build + push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "nchammas (via GitHub)" <gi...@apache.org>.
nchammas commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4938418417
Just wanted to double check one thing with you before you merge:
> Confirm ASF destination config with committer.
Is my analysis about `site` correct? We don't have a staging deployment, so I can't test this out.
In any case, I'll be watching closely here after you merge to make sure everything builds and deploys correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org
Re: [PR] Add workflow to automatically generate and commit site HTML [spark-website]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #697:
URL: https://github.com/apache/spark-website/pull/697#issuecomment-4937499669
LGTM if CI is green
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org