You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by Steve Lawrence <sl...@apache.org> on 2025/02/27 12:44:20 UTC
[DISCUSS] Automated Release Signing
ASF INFRA allows for automated release signing via GitHub actions, as described
here:
https://infra.apache.org/release-signing.html#automated-release-signing
In the past few weeks, a number of changes and pull requests have been created
and/or merged in our various projects and we've added the new
github.com/apache/daffodil-infrastructure repository that adds tools/utilities
that help us meet the listed requirements. Specifically:
* All artifacts being signed can be built reproducibly
There are two outstanding pull requests that are soon to be merged that will
resolve the remaining issues preventing reproducible builds. Additionally, the
new daffodil-infrastructure repo defines a container with an environment similar
to GitHub actions that allows for locally creating reproducible builds:
https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-release
* CI deploys the artifacts to a staging environment
The new daffodil-infrastructure repo includes a custom GitHub action that will
be used by our projects to easily build artifacts in GitHub actions and deploy
them to the ASF staging environment (i.e. repository.apache.org and
dist.apache.org/repos/dist/dev)
https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-candidate
I have also created draft pull requests in the daffodil, daffodil-vscode, and
daffodil-sbt repositories that use this action to build release candidates:
https://github.com/apache/daffodil/pull/1445
https://github.com/apache/daffodil-sbt/pull/94
https://github.com/apache/daffodil-vscode/pull/1184
(Note that these PR's are drafts and may have minor changes before merging)
* The release procedure contains a validation step where all artifacts are
reproduced on trusted hardware before publication to pages intended for end users
A new wiki page has been created that defines the steps and scripts to run to
verify signatures, checksums, and build reproducibility, making use of the
previously mentioned container.
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
These steps use the previously mentioned container to locally build artifacts
and a script to verify checksums, signatures, and reproducibility:
https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-release.sh
The new tag-based release workflow using the above actions has been created here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
This workflow specifies that a VOTE cannot pass unless reproducibility has been
verified.
Note that this new proposed release workflow is virtually identical to the
existing workflow, except that the "Creating a Release Candidate" section is now
simplified to just creating and pushing a single tag. That tag triggers the new
GitHub actions to do all the necessary steps to build, sign, checksum, and stage
artifacts in preparation for a release vote.
I ask that you look over this proposed workflow and see if there are any
issues/concerns/questions. If there are no issues, please respond with a +1 so
there is evidence of consensus for this change.
Once there is agreement, the next step is to begin a conversation with the ASF
Security Team to approve this proposed workflow and work with INFRA to create
signing keys and add them to our projects.
Re: [DISCUSS] Automated Release Signing
Posted by Steve Lawrence <sl...@apache.org>.
Good idea. I've opened a PR to change the check-release script to a container,
and update the release verification wiki to describe how to use this container,
or optionally use the script if the host has the tools available.
On 2025-03-07 11:18 AM, Adams, Joshua wrote:
> +1 with a minor process improvement: Use a container for the check-release.sh script.
>
> My system (Rocky 9) does not have easy access to the msidiff tool and had a different wget version that didn't like the --force-progress option.
>
> Looking forward to the more automated release process.
>
> Josh
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Thursday, February 27, 2025 7:44 AM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: [DISCUSS] Automated Release Signing
>
> ASF INFRA allows for automated release signing via GitHub actions, as described
> here:
>
> https://infra.apache.org/release-signing.html#automated-release-signing
>
> In the past few weeks, a number of changes and pull requests have been created
> and/or merged in our various projects and we've added the new
> github.com/apache/daffodil-infrastructure repository that adds tools/utilities
> that help us meet the listed requirements. Specifically:
>
> * All artifacts being signed can be built reproducibly
>
> There are two outstanding pull requests that are soon to be merged that will
> resolve the remaining issues preventing reproducible builds. Additionally, the
> new daffodil-infrastructure repo defines a container with an environment similar
> to GitHub actions that allows for locally creating reproducible builds:
>
> https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-release
>
> * CI deploys the artifacts to a staging environment
>
> The new daffodil-infrastructure repo includes a custom GitHub action that will
> be used by our projects to easily build artifacts in GitHub actions and deploy
> them to the ASF staging environment (i.e. repository.apache.org and
> dist.apache.org/repos/dist/dev)
>
> https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-candidate
>
> I have also created draft pull requests in the daffodil, daffodil-vscode, and
> daffodil-sbt repositories that use this action to build release candidates:
>
> https://github.com/apache/daffodil/pull/1445
>
> https://github.com/apache/daffodil-sbt/pull/94
>
> https://github.com/apache/daffodil-vscode/pull/1184
>
> (Note that these PR's are drafts and may have minor changes before merging)
>
> * The release procedure contains a validation step where all artifacts are
> reproduced on trusted hardware before publication to pages intended for end users
>
> A new wiki page has been created that defines the steps and scripts to run to
> verify signatures, checksums, and build reproducibility, making use of the
> previously mentioned container.
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
>
> These steps use the previously mentioned container to locally build artifacts
> and a script to verify checksums, signatures, and reproducibility:
>
> https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-release.sh
>
> The new tag-based release workflow using the above actions has been created here:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
>
> This workflow specifies that a VOTE cannot pass unless reproducibility has been
> verified.
>
> Note that this new proposed release workflow is virtually identical to the
> existing workflow, except that the "Creating a Release Candidate" section is now
> simplified to just creating and pushing a single tag. That tag triggers the new
> GitHub actions to do all the necessary steps to build, sign, checksum, and stage
> artifacts in preparation for a release vote.
>
>
> I ask that you look over this proposed workflow and see if there are any
> issues/concerns/questions. If there are no issues, please respond with a +1 so
> there is evidence of consensus for this change.
>
> Once there is agreement, the next step is to begin a conversation with the ASF
> Security Team to approve this proposed workflow and work with INFRA to create
> signing keys and add them to our projects.
>
Re: [DISCUSS] Automated Release Signing
Posted by "Adams, Joshua" <ja...@owlcyberdefense.com.INVALID>.
+1 with a minor process improvement: Use a container for the check-release.sh script.
My system (Rocky 9) does not have easy access to the msidiff tool and had a different wget version that didn't like the --force-progress option.
Looking forward to the more automated release process.
Josh
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Thursday, February 27, 2025 7:44 AM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: [DISCUSS] Automated Release Signing
ASF INFRA allows for automated release signing via GitHub actions, as described
here:
https://infra.apache.org/release-signing.html#automated-release-signing
In the past few weeks, a number of changes and pull requests have been created
and/or merged in our various projects and we've added the new
github.com/apache/daffodil-infrastructure repository that adds tools/utilities
that help us meet the listed requirements. Specifically:
* All artifacts being signed can be built reproducibly
There are two outstanding pull requests that are soon to be merged that will
resolve the remaining issues preventing reproducible builds. Additionally, the
new daffodil-infrastructure repo defines a container with an environment similar
to GitHub actions that allows for locally creating reproducible builds:
https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-release
* CI deploys the artifacts to a staging environment
The new daffodil-infrastructure repo includes a custom GitHub action that will
be used by our projects to easily build artifacts in GitHub actions and deploy
them to the ASF staging environment (i.e. repository.apache.org and
dist.apache.org/repos/dist/dev)
https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-candidate
I have also created draft pull requests in the daffodil, daffodil-vscode, and
daffodil-sbt repositories that use this action to build release candidates:
https://github.com/apache/daffodil/pull/1445
https://github.com/apache/daffodil-sbt/pull/94
https://github.com/apache/daffodil-vscode/pull/1184
(Note that these PR's are drafts and may have minor changes before merging)
* The release procedure contains a validation step where all artifacts are
reproduced on trusted hardware before publication to pages intended for end users
A new wiki page has been created that defines the steps and scripts to run to
verify signatures, checksums, and build reproducibility, making use of the
previously mentioned container.
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
These steps use the previously mentioned container to locally build artifacts
and a script to verify checksums, signatures, and reproducibility:
https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-release.sh
The new tag-based release workflow using the above actions has been created here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
This workflow specifies that a VOTE cannot pass unless reproducibility has been
verified.
Note that this new proposed release workflow is virtually identical to the
existing workflow, except that the "Creating a Release Candidate" section is now
simplified to just creating and pushing a single tag. That tag triggers the new
GitHub actions to do all the necessary steps to build, sign, checksum, and stage
artifacts in preparation for a release vote.
I ask that you look over this proposed workflow and see if there are any
issues/concerns/questions. If there are no issues, please respond with a +1 so
there is evidence of consensus for this change.
Once there is agreement, the next step is to begin a conversation with the ASF
Security Team to approve this proposed workflow and work with INFRA to create
signing keys and add them to our projects.
Re: [DISCUSS] Automated Release Signing
Posted by "Kilo, Olabusayo" <ok...@owlcyberdefense.com.INVALID>.
+1
Contents look good, though I didn't actually run anything.
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Thursday, February 27, 2025 7:44 AM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: [DISCUSS] Automated Release Signing
ASF INFRA allows for automated release signing via GitHub actions, as described
here:
https://infra.apache.org/release-signing.html#automated-release-signing
In the past few weeks, a number of changes and pull requests have been created
and/or merged in our various projects and we've added the new
github.com/apache/daffodil-infrastructure repository that adds tools/utilities
that help us meet the listed requirements. Specifically:
* All artifacts being signed can be built reproducibly
There are two outstanding pull requests that are soon to be merged that will
resolve the remaining issues preventing reproducible builds. Additionally, the
new daffodil-infrastructure repo defines a container with an environment similar
to GitHub actions that allows for locally creating reproducible builds:
https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-release
* CI deploys the artifacts to a staging environment
The new daffodil-infrastructure repo includes a custom GitHub action that will
be used by our projects to easily build artifacts in GitHub actions and deploy
them to the ASF staging environment (i.e. repository.apache.org and
dist.apache.org/repos/dist/dev)
https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-candidate
I have also created draft pull requests in the daffodil, daffodil-vscode, and
daffodil-sbt repositories that use this action to build release candidates:
https://github.com/apache/daffodil/pull/1445
https://github.com/apache/daffodil-sbt/pull/94
https://github.com/apache/daffodil-vscode/pull/1184
(Note that these PR's are drafts and may have minor changes before merging)
* The release procedure contains a validation step where all artifacts are
reproduced on trusted hardware before publication to pages intended for end users
A new wiki page has been created that defines the steps and scripts to run to
verify signatures, checksums, and build reproducibility, making use of the
previously mentioned container.
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
These steps use the previously mentioned container to locally build artifacts
and a script to verify checksums, signatures, and reproducibility:
https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-release.sh
The new tag-based release workflow using the above actions has been created here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
This workflow specifies that a VOTE cannot pass unless reproducibility has been
verified.
Note that this new proposed release workflow is virtually identical to the
existing workflow, except that the "Creating a Release Candidate" section is now
simplified to just creating and pushing a single tag. That tag triggers the new
GitHub actions to do all the necessary steps to build, sign, checksum, and stage
artifacts in preparation for a release vote.
I ask that you look over this proposed workflow and see if there are any
issues/concerns/questions. If there are no issues, please respond with a +1 so
there is evidence of consensus for this change.
Once there is agreement, the next step is to begin a conversation with the ASF
Security Team to approve this proposed workflow and work with INFRA to create
signing keys and add them to our projects.
Re: [DISCUSS] Automated Release Signing
Posted by Mike Beckerle <mb...@apache.org>.
+1
On Thu, Feb 27, 2025 at 7:45 AM Steve Lawrence <sl...@apache.org> wrote:
> ASF INFRA allows for automated release signing via GitHub actions, as
> described
> here:
>
> https://infra.apache.org/release-signing.html#automated-release-signing
>
> In the past few weeks, a number of changes and pull requests have been
> created
> and/or merged in our various projects and we've added the new
> github.com/apache/daffodil-infrastructure repository that adds
> tools/utilities
> that help us meet the listed requirements. Specifically:
>
> * All artifacts being signed can be built reproducibly
>
> There are two outstanding pull requests that are soon to be merged that
> will
> resolve the remaining issues preventing reproducible builds. Additionally,
> the
> new daffodil-infrastructure repo defines a container with an environment
> similar
> to GitHub actions that allows for locally creating reproducible builds:
>
>
> https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-release
>
> * CI deploys the artifacts to a staging environment
>
> The new daffodil-infrastructure repo includes a custom GitHub action that
> will
> be used by our projects to easily build artifacts in GitHub actions and
> deploy
> them to the ASF staging environment (i.e. repository.apache.org and
> dist.apache.org/repos/dist/dev)
>
>
> https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-candidate
>
> I have also created draft pull requests in the daffodil, daffodil-vscode,
> and
> daffodil-sbt repositories that use this action to build release candidates:
>
> https://github.com/apache/daffodil/pull/1445
>
> https://github.com/apache/daffodil-sbt/pull/94
>
> https://github.com/apache/daffodil-vscode/pull/1184
>
> (Note that these PR's are drafts and may have minor changes before merging)
>
> * The release procedure contains a validation step where all artifacts are
> reproduced on trusted hardware before publication to pages intended for
> end users
>
> A new wiki page has been created that defines the steps and scripts to run
> to
> verify signatures, checksums, and build reproducibility, making use of the
> previously mentioned container.
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
>
> These steps use the previously mentioned container to locally build
> artifacts
> and a script to verify checksums, signatures, and reproducibility:
>
>
> https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-release.sh
>
> The new tag-based release workflow using the above actions has been
> created here:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
>
> This workflow specifies that a VOTE cannot pass unless reproducibility has
> been
> verified.
>
> Note that this new proposed release workflow is virtually identical to the
> existing workflow, except that the "Creating a Release Candidate" section
> is now
> simplified to just creating and pushing a single tag. That tag triggers
> the new
> GitHub actions to do all the necessary steps to build, sign, checksum, and
> stage
> artifacts in preparation for a release vote.
>
>
> I ask that you look over this proposed workflow and see if there are any
> issues/concerns/questions. If there are no issues, please respond with a
> +1 so
> there is evidence of consensus for this change.
>
> Once there is agreement, the next step is to begin a conversation with the
> ASF
> Security Team to approve this proposed workflow and work with INFRA to
> create
> signing keys and add them to our projects.
>
Re: [DISCUSS] Automated Release Signing
Posted by Steve Lawrence <sl...@apache.org>.
Thanks to every that took at look at this. I'll start the discussions with ASF
Infra and Security to set things up.
Note that the check-release script has been moved to a container, so the link in
the first email is broken. The new link is:
https://github.com/apache/daffodil-infrastructure/blob/main/containers/check-release/src/check-release.sh
On 2025-02-27 07:44 AM, Steve Lawrence wrote:
> ASF INFRA allows for automated release signing via GitHub actions, as described
> here:
>
> https://infra.apache.org/release-signing.html#automated-release-signing
>
> In the past few weeks, a number of changes and pull requests have been created
> and/or merged in our various projects and we've added the new github.com/apache/
> daffodil-infrastructure repository that adds tools/utilities that help us meet
> the listed requirements. Specifically:
>
> * All artifacts being signed can be built reproducibly
>
> There are two outstanding pull requests that are soon to be merged that will
> resolve the remaining issues preventing reproducible builds. Additionally, the
> new daffodil-infrastructure repo defines a container with an environment similar
> to GitHub actions that allows for locally creating reproducible builds:
>
> https://github.com/apache/daffodil-infrastructure/tree/main/containers/build-
> release
>
> * CI deploys the artifacts to a staging environment
>
> The new daffodil-infrastructure repo includes a custom GitHub action that will
> be used by our projects to easily build artifacts in GitHub actions and deploy
> them to the ASF staging environment (i.e. repository.apache.org and
> dist.apache.org/repos/dist/dev)
>
> https://github.com/apache/daffodil-infrastructure/tree/main/actions/release-
> candidate
>
> I have also created draft pull requests in the daffodil, daffodil-vscode, and
> daffodil-sbt repositories that use this action to build release candidates:
>
> https://github.com/apache/daffodil/pull/1445
>
> https://github.com/apache/daffodil-sbt/pull/94
>
> https://github.com/apache/daffodil-vscode/pull/1184
>
> (Note that these PR's are drafts and may have minor changes before merging)
>
> * The release procedure contains a validation step where all artifacts are
> reproduced on trusted hardware before publication to pages intended for end users
>
> A new wiki page has been created that defines the steps and scripts to run to
> verify signatures, checksums, and build reproducibility, making use of the
> previously mentioned container.
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Verification
>
> These steps use the previously mentioned container to locally build artifacts
> and a script to verify checksums, signatures, and reproducibility:
>
> https://github.com/apache/daffodil-infrastructure/blob/main/scripts/check-
> release.sh
>
> The new tag-based release workflow using the above actions has been created here:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=340037483
>
> This workflow specifies that a VOTE cannot pass unless reproducibility has been
> verified.
>
> Note that this new proposed release workflow is virtually identical to the
> existing workflow, except that the "Creating a Release Candidate" section is now
> simplified to just creating and pushing a single tag. That tag triggers the new
> GitHub actions to do all the necessary steps to build, sign, checksum, and stage
> artifacts in preparation for a release vote.
>
>
> I ask that you look over this proposed workflow and see if there are any issues/
> concerns/questions. If there are no issues, please respond with a +1 so there is
> evidence of consensus for this change.
>
> Once there is agreement, the next step is to begin a conversation with the ASF
> Security Team to approve this proposed workflow and work with INFRA to create
> signing keys and add them to our projects.