You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Nick Dimiduk <nd...@apache.org> on 2026/01/15 14:35:50 UTC
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Hi team,
I've added GitHub Actions to run Yetus general checks directly on pull requests. This has been backported to all active branches (master, branch-3, branch-2, branch-2.6, branch-2.5).
What's changing:
- PRs will now trigger a "Yetus General Check" workflow that runs checkstyle, javac, pylint, shellcheck, and other static analysis checks
- Results appear as GitHub status checks and PR comments (same as Jenkins)
- Old bot comments from previous runs are automatically hidden to reduce noise
- Uses the same scripts, Dockerfile, &c. that we use for Jenkins
What's NOT changing:
- Jenkins precommit builds continue to run as before
- Your PR workflow remains the same
- Unit/integration tests still run on Jenkins, not GitHub Actions
JIRAs: HBASE-29787, HBASE-29829
Give us a shout if anything breaks for you. Also, feel free to pick up migrating other jobs, if you want to (I've not filed any subsequent tickets). It's a little bit tedious and it seems that you need push rights to the origin github repo to introduce a new workflow (I'm not sure about editing an existing workflow).
Thanks,
Nick
On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> Heya,
>
> I'd like to take the community temperature on migrating our build infra
> from the ci-hbase.a.o Jenkins instance to something built on GitHub
> Actions. I have several reasons that justify this proposal.
>
> As some of you may know, our community funding has reduced and we will no
> longer be able to sustain the current fleet of build infrastructure. So,
> one motivation for this proposal is cost-cutting: I think that we'll be
> able to operate at lower costs if we can migrate to a provisioned-as-needed
> model of consumption.
>
> My second reason is an optimistic appeal to a larger contributor base. I
> suspect that if we can modernize our infrastructure then we will increase
> the pool of contributors who might be able to participate in this area. I
> believe that GH Actions (and systems like it) is more prevalent in the
> industry than Jenkins, which means that more people already have experience
> with the platform and more people will feel compelled to offer support to
> an OSS project that uses the platform as a means of growing their own
> skillset and as a means of bolstering their CVs.
>
> Dove-tailed into reason two is reason three: I believe that there is a
> large community of folks who are developing GitHub Actions on its
> marketplace. We would effectively open ourselves up to more off-the-shelf
> offerings and those offerings would be in our hands directly. By contrast,
> I don't think there's as much development in Jenkins plugins, and the
> process of adding a new plugin to our Jenkins instance requires filing an
> INFRA ticket.
>
> These are my motivations. I'm still not clear on what's possible yet for
> ASF projects. I have filed an INFRA ticket, requesting whatever is
> necessary for us to start an experiment. Indeed, I believe that there are
> some major limitations on the current implementation provided by the ASF,
> and as far as I can tell, only one project with a build footprint that
> resembles HBase has pursued this effort. I've catalogued the applicable
> information that I've found so far on that issue.
>
> https://issues.apache.org/jira/browse/INFRA-26170
>
> Thanks,
> Nick
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
I've opened HBASER-29872 for tracking the disabling of our previous
pre commit github jenkins job.
And there is a difference that, now if a PR is not opened by a
committer, the pre commit checks workflow needs to be approved by a
committer before running, while in the past the jenkins job would scan
the github PRs and schedule pre commit jobs automatically.
Nick Dimiduk <nd...@apache.org> 于2026年2月6日周五 23:45写道：
>
> Hi Charles,
>
> I did, yes. My strategy is to break the large test set down into several
> waves. Actually, I have 5 total waves for unit tests: small, medium,
> large-1, large-2, large-3. They run in parallel and so the total time for
> feedback is much faster than the Jenkins model... at the cost of pretty
> high resource usage on branch-2*. It's possible that we'll run into issues
> with Github's generosity with the free tier runners, but we'll have to
> cross that bridge with INFRA if/when it comes up.
>
> -n
>
> On Fri, Feb 6, 2026 at 4:39 PM Charles Connell via dev <de...@hbase.apache.org>
> wrote:
>
> > Thanks for your work Nick,
> >
> > When exploring Github actions myself, I found that they might struggle
> > to run the unit tests in the hbase-server module. Any single action
> > step is limited to 6 hours runtime, and running these tests can take
> > longer than that on the action runner, in my experience. Did you work
> > around this somehow?
> >
> > Charles
> >
> > On Fri, Feb 6, 2026 at 10:29 AM Nick Dimiduk <nd...@apache.org> wrote:
> > >
> > > Heya team,
> > >
> > > We now have static analysis, build, and unit test coverage via GHA
> > implemented for PRs against all branches. In the absence of comments,
> > you'll find the detailed Yetus table reports in the "Action Summary"
> > section of the checks interface. Implementation details are attached to
> > HBASE-29838. As before, shout if you find something broken.
> > >
> > > As of this effort, all of our PR work is being done in GHA. All of the
> > Jenkins infrastructure is still present and intact, but I have disabled the
> > job in the Jenkins UI [0]. If you're having problems with the new tooling
> > and want to restore the Jenkins job, just flip that switch back to enabled.
> > >
> > > There's more jobs to be converted, I've not filed any additional Jiras
> > yet. Please feel encouraged to tackle something if you're feeling so
> > inclined.
> > >
> > > Thanks,
> > > Nick
> > >
> > > [0]: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR
> > >
> > > On 2026/01/15 14:35:50 Nick Dimiduk wrote:
> > > > Hi team,
> > > >
> > > > I've added GitHub Actions to run Yetus general checks directly on pull
> > requests. This has been backported to all active branches (master,
> > branch-3, branch-2, branch-2.6, branch-2.5).
> > > >
> > > > What's changing:
> > > > - PRs will now trigger a "Yetus General Check" workflow that runs
> > checkstyle, javac, pylint, shellcheck, and other static analysis checks
> > > > - Results appear as GitHub status checks and PR comments (same as
> > Jenkins)
> > > > - Old bot comments from previous runs are automatically hidden to
> > reduce noise
> > > > - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> > > >
> > > > What's NOT changing:
> > > > - Jenkins precommit builds continue to run as before
> > > > - Your PR workflow remains the same
> > > > - Unit/integration tests still run on Jenkins, not GitHub Actions
> > > >
> > > > JIRAs: HBASE-29787, HBASE-29829
> > > >
> > > > Give us a shout if anything breaks for you. Also, feel free to pick up
> > migrating other jobs, if you want to (I've not filed any subsequent
> > tickets). It's a little bit tedious and it seems that you need push rights
> > to the origin github repo to introduce a new workflow (I'm not sure about
> > editing an existing workflow).
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > > > On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > > > > Heya,
> > > > >
> > > > > I'd like to take the community temperature on migrating our build
> > infra
> > > > > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > > > > Actions. I have several reasons that justify this proposal.
> > > > >
> > > > > As some of you may know, our community funding has reduced and we
> > will no
> > > > > longer be able to sustain the current fleet of build infrastructure.
> > So,
> > > > > one motivation for this proposal is cost-cutting: I think that we'll
> > be
> > > > > able to operate at lower costs if we can migrate to a
> > provisioned-as-needed
> > > > > model of consumption.
> > > > >
> > > > > My second reason is an optimistic appeal to a larger contributor
> > base. I
> > > > > suspect that if we can modernize our infrastructure then we will
> > increase
> > > > > the pool of contributors who might be able to participate in this
> > area. I
> > > > > believe that GH Actions (and systems like it) is more prevalent in
> > the
> > > > > industry than Jenkins, which means that more people already have
> > experience
> > > > > with the platform and more people will feel compelled to offer
> > support to
> > > > > an OSS project that uses the platform as a means of growing their own
> > > > > skillset and as a means of bolstering their CVs.
> > > > >
> > > > > Dove-tailed into reason two is reason three: I believe that there is
> > a
> > > > > large community of folks who are developing GitHub Actions on its
> > > > > marketplace. We would effectively open ourselves up to more
> > off-the-shelf
> > > > > offerings and those offerings would be in our hands directly. By
> > contrast,
> > > > > I don't think there's as much development in Jenkins plugins, and the
> > > > > process of adding a new plugin to our Jenkins instance requires
> > filing an
> > > > > INFRA ticket.
> > > > >
> > > > > These are my motivations. I'm still not clear on what's possible yet
> > for
> > > > > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > > > > necessary for us to start an experiment. Indeed, I believe that
> > there are
> > > > > some major limitations on the current implementation provided by the
> > ASF,
> > > > > and as far as I can tell, only one project with a build footprint
> > that
> > > > > resembles HBase has pursued this effort. I've catalogued the
> > applicable
> > > > > information that I've found so far on that issue.
> > > > >
> > > > > https://issues.apache.org/jira/browse/INFRA-26170
> > > > >
> > > > > Thanks,
> > > > > Nick
> > > > >
> > > >
> >
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
Hi Charles,
I did, yes. My strategy is to break the large test set down into several
waves. Actually, I have 5 total waves for unit tests: small, medium,
large-1, large-2, large-3. They run in parallel and so the total time for
feedback is much faster than the Jenkins model... at the cost of pretty
high resource usage on branch-2*. It's possible that we'll run into issues
with Github's generosity with the free tier runners, but we'll have to
cross that bridge with INFRA if/when it comes up.
-n
On Fri, Feb 6, 2026 at 4:39 PM Charles Connell via dev <de...@hbase.apache.org>
wrote:
> Thanks for your work Nick,
>
> When exploring Github actions myself, I found that they might struggle
> to run the unit tests in the hbase-server module. Any single action
> step is limited to 6 hours runtime, and running these tests can take
> longer than that on the action runner, in my experience. Did you work
> around this somehow?
>
> Charles
>
> On Fri, Feb 6, 2026 at 10:29 AM Nick Dimiduk <nd...@apache.org> wrote:
> >
> > Heya team,
> >
> > We now have static analysis, build, and unit test coverage via GHA
> implemented for PRs against all branches. In the absence of comments,
> you'll find the detailed Yetus table reports in the "Action Summary"
> section of the checks interface. Implementation details are attached to
> HBASE-29838. As before, shout if you find something broken.
> >
> > As of this effort, all of our PR work is being done in GHA. All of the
> Jenkins infrastructure is still present and intact, but I have disabled the
> job in the Jenkins UI [0]. If you're having problems with the new tooling
> and want to restore the Jenkins job, just flip that switch back to enabled.
> >
> > There's more jobs to be converted, I've not filed any additional Jiras
> yet. Please feel encouraged to tackle something if you're feeling so
> inclined.
> >
> > Thanks,
> > Nick
> >
> > [0]: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR
> >
> > On 2026/01/15 14:35:50 Nick Dimiduk wrote:
> > > Hi team,
> > >
> > > I've added GitHub Actions to run Yetus general checks directly on pull
> requests. This has been backported to all active branches (master,
> branch-3, branch-2, branch-2.6, branch-2.5).
> > >
> > > What's changing:
> > > - PRs will now trigger a "Yetus General Check" workflow that runs
> checkstyle, javac, pylint, shellcheck, and other static analysis checks
> > > - Results appear as GitHub status checks and PR comments (same as
> Jenkins)
> > > - Old bot comments from previous runs are automatically hidden to
> reduce noise
> > > - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> > >
> > > What's NOT changing:
> > > - Jenkins precommit builds continue to run as before
> > > - Your PR workflow remains the same
> > > - Unit/integration tests still run on Jenkins, not GitHub Actions
> > >
> > > JIRAs: HBASE-29787, HBASE-29829
> > >
> > > Give us a shout if anything breaks for you. Also, feel free to pick up
> migrating other jobs, if you want to (I've not filed any subsequent
> tickets). It's a little bit tedious and it seems that you need push rights
> to the origin github repo to introduce a new workflow (I'm not sure about
> editing an existing workflow).
> > >
> > > Thanks,
> > > Nick
> > >
> > > On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > > > Heya,
> > > >
> > > > I'd like to take the community temperature on migrating our build
> infra
> > > > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > > > Actions. I have several reasons that justify this proposal.
> > > >
> > > > As some of you may know, our community funding has reduced and we
> will no
> > > > longer be able to sustain the current fleet of build infrastructure.
> So,
> > > > one motivation for this proposal is cost-cutting: I think that we'll
> be
> > > > able to operate at lower costs if we can migrate to a
> provisioned-as-needed
> > > > model of consumption.
> > > >
> > > > My second reason is an optimistic appeal to a larger contributor
> base. I
> > > > suspect that if we can modernize our infrastructure then we will
> increase
> > > > the pool of contributors who might be able to participate in this
> area. I
> > > > believe that GH Actions (and systems like it) is more prevalent in
> the
> > > > industry than Jenkins, which means that more people already have
> experience
> > > > with the platform and more people will feel compelled to offer
> support to
> > > > an OSS project that uses the platform as a means of growing their own
> > > > skillset and as a means of bolstering their CVs.
> > > >
> > > > Dove-tailed into reason two is reason three: I believe that there is
> a
> > > > large community of folks who are developing GitHub Actions on its
> > > > marketplace. We would effectively open ourselves up to more
> off-the-shelf
> > > > offerings and those offerings would be in our hands directly. By
> contrast,
> > > > I don't think there's as much development in Jenkins plugins, and the
> > > > process of adding a new plugin to our Jenkins instance requires
> filing an
> > > > INFRA ticket.
> > > >
> > > > These are my motivations. I'm still not clear on what's possible yet
> for
> > > > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > > > necessary for us to start an experiment. Indeed, I believe that
> there are
> > > > some major limitations on the current implementation provided by the
> ASF,
> > > > and as far as I can tell, only one project with a build footprint
> that
> > > > resembles HBase has pursued this effort. I've catalogued the
> applicable
> > > > information that I've found so far on that issue.
> > > >
> > > > https://issues.apache.org/jira/browse/INFRA-26170
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Charles Connell via dev <de...@hbase.apache.org>.
Thanks for your work Nick,
When exploring Github actions myself, I found that they might struggle
to run the unit tests in the hbase-server module. Any single action
step is limited to 6 hours runtime, and running these tests can take
longer than that on the action runner, in my experience. Did you work
around this somehow?
Charles
On Fri, Feb 6, 2026 at 10:29 AM Nick Dimiduk <nd...@apache.org> wrote:
>
> Heya team,
>
> We now have static analysis, build, and unit test coverage via GHA implemented for PRs against all branches. In the absence of comments, you'll find the detailed Yetus table reports in the "Action Summary" section of the checks interface. Implementation details are attached to HBASE-29838. As before, shout if you find something broken.
>
> As of this effort, all of our PR work is being done in GHA. All of the Jenkins infrastructure is still present and intact, but I have disabled the job in the Jenkins UI [0]. If you're having problems with the new tooling and want to restore the Jenkins job, just flip that switch back to enabled.
>
> There's more jobs to be converted, I've not filed any additional Jiras yet. Please feel encouraged to tackle something if you're feeling so inclined.
>
> Thanks,
> Nick
>
> [0]: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR
>
> On 2026/01/15 14:35:50 Nick Dimiduk wrote:
> > Hi team,
> >
> > I've added GitHub Actions to run Yetus general checks directly on pull requests. This has been backported to all active branches (master, branch-3, branch-2, branch-2.6, branch-2.5).
> >
> > What's changing:
> > - PRs will now trigger a "Yetus General Check" workflow that runs checkstyle, javac, pylint, shellcheck, and other static analysis checks
> > - Results appear as GitHub status checks and PR comments (same as Jenkins)
> > - Old bot comments from previous runs are automatically hidden to reduce noise
> > - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> >
> > What's NOT changing:
> > - Jenkins precommit builds continue to run as before
> > - Your PR workflow remains the same
> > - Unit/integration tests still run on Jenkins, not GitHub Actions
> >
> > JIRAs: HBASE-29787, HBASE-29829
> >
> > Give us a shout if anything breaks for you. Also, feel free to pick up migrating other jobs, if you want to (I've not filed any subsequent tickets). It's a little bit tedious and it seems that you need push rights to the origin github repo to introduce a new workflow (I'm not sure about editing an existing workflow).
> >
> > Thanks,
> > Nick
> >
> > On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > > Heya,
> > >
> > > I'd like to take the community temperature on migrating our build infra
> > > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > > Actions. I have several reasons that justify this proposal.
> > >
> > > As some of you may know, our community funding has reduced and we will no
> > > longer be able to sustain the current fleet of build infrastructure. So,
> > > one motivation for this proposal is cost-cutting: I think that we'll be
> > > able to operate at lower costs if we can migrate to a provisioned-as-needed
> > > model of consumption.
> > >
> > > My second reason is an optimistic appeal to a larger contributor base. I
> > > suspect that if we can modernize our infrastructure then we will increase
> > > the pool of contributors who might be able to participate in this area. I
> > > believe that GH Actions (and systems like it) is more prevalent in the
> > > industry than Jenkins, which means that more people already have experience
> > > with the platform and more people will feel compelled to offer support to
> > > an OSS project that uses the platform as a means of growing their own
> > > skillset and as a means of bolstering their CVs.
> > >
> > > Dove-tailed into reason two is reason three: I believe that there is a
> > > large community of folks who are developing GitHub Actions on its
> > > marketplace. We would effectively open ourselves up to more off-the-shelf
> > > offerings and those offerings would be in our hands directly. By contrast,
> > > I don't think there's as much development in Jenkins plugins, and the
> > > process of adding a new plugin to our Jenkins instance requires filing an
> > > INFRA ticket.
> > >
> > > These are my motivations. I'm still not clear on what's possible yet for
> > > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > > necessary for us to start an experiment. Indeed, I believe that there are
> > > some major limitations on the current implementation provided by the ASF,
> > > and as far as I can tell, only one project with a build footprint that
> > > resembles HBase has pursued this effort. I've catalogued the applicable
> > > information that I've found so far on that issue.
> > >
> > > https://issues.apache.org/jira/browse/INFRA-26170
> > >
> > > Thanks,
> > > Nick
> > >
> >
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
Heya team,
We now have static analysis, build, and unit test coverage via GHA implemented for PRs against all branches. In the absence of comments, you'll find the detailed Yetus table reports in the "Action Summary" section of the checks interface. Implementation details are attached to HBASE-29838. As before, shout if you find something broken.
As of this effort, all of our PR work is being done in GHA. All of the Jenkins infrastructure is still present and intact, but I have disabled the job in the Jenkins UI [0]. If you're having problems with the new tooling and want to restore the Jenkins job, just flip that switch back to enabled.
There's more jobs to be converted, I've not filed any additional Jiras yet. Please feel encouraged to tackle something if you're feeling so inclined.
Thanks,
Nick
[0]: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR
On 2026/01/15 14:35:50 Nick Dimiduk wrote:
> Hi team,
>
> I've added GitHub Actions to run Yetus general checks directly on pull requests. This has been backported to all active branches (master, branch-3, branch-2, branch-2.6, branch-2.5).
>
> What's changing:
> - PRs will now trigger a "Yetus General Check" workflow that runs checkstyle, javac, pylint, shellcheck, and other static analysis checks
> - Results appear as GitHub status checks and PR comments (same as Jenkins)
> - Old bot comments from previous runs are automatically hidden to reduce noise
> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
>
> What's NOT changing:
> - Jenkins precommit builds continue to run as before
> - Your PR workflow remains the same
> - Unit/integration tests still run on Jenkins, not GitHub Actions
>
> JIRAs: HBASE-29787, HBASE-29829
>
> Give us a shout if anything breaks for you. Also, feel free to pick up migrating other jobs, if you want to (I've not filed any subsequent tickets). It's a little bit tedious and it seems that you need push rights to the origin github repo to introduce a new workflow (I'm not sure about editing an existing workflow).
>
> Thanks,
> Nick
>
> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > Heya,
> >
> > I'd like to take the community temperature on migrating our build infra
> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > Actions. I have several reasons that justify this proposal.
> >
> > As some of you may know, our community funding has reduced and we will no
> > longer be able to sustain the current fleet of build infrastructure. So,
> > one motivation for this proposal is cost-cutting: I think that we'll be
> > able to operate at lower costs if we can migrate to a provisioned-as-needed
> > model of consumption.
> >
> > My second reason is an optimistic appeal to a larger contributor base. I
> > suspect that if we can modernize our infrastructure then we will increase
> > the pool of contributors who might be able to participate in this area. I
> > believe that GH Actions (and systems like it) is more prevalent in the
> > industry than Jenkins, which means that more people already have experience
> > with the platform and more people will feel compelled to offer support to
> > an OSS project that uses the platform as a means of growing their own
> > skillset and as a means of bolstering their CVs.
> >
> > Dove-tailed into reason two is reason three: I believe that there is a
> > large community of folks who are developing GitHub Actions on its
> > marketplace. We would effectively open ourselves up to more off-the-shelf
> > offerings and those offerings would be in our hands directly. By contrast,
> > I don't think there's as much development in Jenkins plugins, and the
> > process of adding a new plugin to our Jenkins instance requires filing an
> > INFRA ticket.
> >
> > These are my motivations. I'm still not clear on what's possible yet for
> > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > necessary for us to start an experiment. Indeed, I believe that there are
> > some major limitations on the current implementation provided by the ASF,
> > and as far as I can tell, only one project with a build footprint that
> > resembles HBase has pursued this effort. I've catalogued the applicable
> > information that I've found so far on that issue.
> >
> > https://issues.apache.org/jira/browse/INFRA-26170
> >
> > Thanks,
> > Nick
> >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
I did some digging and filed
https://issues.apache.org/jira/browse/YETUS-1261.
On Mon, Feb 9, 2026 at 11:09 AM Xiao Liu <li...@apache.org> wrote:
> Hi, Nick, got it, thank you for your patient explanation!!!
>
> Best
>
> On 2026/02/09 09:40:43 Nick Dimiduk wrote:
> > Hi Xiao Liu,
> >
> > As far as I can tell, our Yetus invocations have always done that, both
> in
> > Jenkins and GHA. I think it’s an artifact of the fact that the plugin was
> > written against an earlier version of the GH permissions model. My
> > understanding is that you no longer need any `repo:* write` permission to
> > report the result of a check.
> >
> > Let take a note to get to the bottom of it.
> >
> > Thanks for taking a close look!
> >
> > -n
> >
> > On Mon, 9 Feb 2026 at 10:28, Xiao Liu <li...@apache.org> wrote:
> >
> > > Hi, Nick Dimiduk. Excellent work!
> > >
> > > I noticed that the current CI prints the error “ERROR: Failed to write
> > > GitHub status. Token expired or missing repo:status write?” during the
> > > Adding GitHub status phase(for example [1]). Is there an issue with
> this
> > > error?
> > >
> > > [1]
> > >
> https://github.com/apache/hbase/actions/runs/21818346708/job/62945085724?pr=7716
> > >
> > > Best
> > >
> > > On 2026/01/17 15:20:41 Nick Dimiduk wrote:
> > > > TL;DR, we cannot have the full features of Jenkins-based checks via
> GHA
> > > > because we cannot grant necessary permissions. This is a restriction
> on
> > > any
> > > > PR posted from an external repo. It’s not an ASF restriction but a GH
> > > > restriction. I didn’t realize this initially because I did initial
> > > > development from a branch off of the apache/hbase repo instead of my
> > > > personal one.
> > > >
> > > > The only way our GHA pre-commit can interact with the PR seems to be
> via
> > > > the Checks API. No more comment with summary of what was run. The
> action
> > > > results are still available in full, I have a step that uploads them
> to
> > > the
> > > > GH archive system. But no longer are there convenient links off to
> the
> > > > reports and logs.
> > > >
> > > > You can see an example of what all this looks like on
> > > > https://github.com/apache/hbase/pull/7639 — look for activity from
> the
> > > GHA
> > > > robot, not from Jenkins (they have different names and icons).
> > > >
> > > > In my opinion, this is an acceptable compromise. I’d like to improve
> > > Yetus
> > > > so that every plugin that currently leaves a +1/-1 in the vote table
> also
> > > > leaves a Check. Currently it seems only -1 results leave a Check.
> > > >
> > > > Any other suggestions or concerns?
> > > >
> > > > Thanks,
> > > > Nick
> > > >
> > > > On Thu, 15 Jan 2026 at 15:50, Nick Dimiduk <nd...@apache.org>
> wrote:
> > > >
> > > > > I spoke a little too soon, there's some issue on non-master
> branches
> > > that
> > > > > didn't show on the back-port PRs. Filed what I know on INFRA-27571.
> > > > >
> > > > > -n
> > > > >
> > > > > On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org>
> > > wrote:
> > > > >
> > > > >> Hi team,
> > > > >>
> > > > >> I've added GitHub Actions to run Yetus general checks directly on
> pull
> > > > >> requests. This has been backported to all active branches (master,
> > > > >> branch-3, branch-2, branch-2.6, branch-2.5).
> > > > >>
> > > > >> What's changing:
> > > > >> - PRs will now trigger a "Yetus General Check" workflow that runs
> > > > >> checkstyle, javac, pylint, shellcheck, and other static analysis
> > > checks
> > > > >> - Results appear as GitHub status checks and PR comments (same as
> > > > >> Jenkins)
> > > > >> - Old bot comments from previous runs are automatically hidden to
> > > reduce
> > > > >> noise
> > > > >> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> > > > >>
> > > > >> What's NOT changing:
> > > > >> - Jenkins precommit builds continue to run as before
> > > > >> - Your PR workflow remains the same
> > > > >> - Unit/integration tests still run on Jenkins, not GitHub Actions
> > > > >>
> > > > >> JIRAs: HBASE-29787, HBASE-29829
> > > > >>
> > > > >> Give us a shout if anything breaks for you. Also, feel free to
> pick up
> > > > >> migrating other jobs, if you want to (I've not filed any
> subsequent
> > > > >> tickets). It's a little bit tedious and it seems that you need
> push
> > > rights
> > > > >> to the origin github repo to introduce a new workflow (I'm not
> sure
> > > about
> > > > >> editing an existing workflow).
> > > > >>
> > > > >> Thanks,
> > > > >> Nick
> > > > >>
> > > > >> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > > > >> > Heya,
> > > > >> >
> > > > >> > I'd like to take the community temperature on migrating our
> build
> > > infra
> > > > >> > from the ci-hbase.a.o Jenkins instance to something built on
> GitHub
> > > > >> > Actions. I have several reasons that justify this proposal.
> > > > >> >
> > > > >> > As some of you may know, our community funding has reduced and
> we
> > > will
> > > > >> no
> > > > >> > longer be able to sustain the current fleet of build
> > > infrastructure. So,
> > > > >> > one motivation for this proposal is cost-cutting: I think that
> > > we'll be
> > > > >> > able to operate at lower costs if we can migrate to a
> > > > >> provisioned-as-needed
> > > > >> > model of consumption.
> > > > >> >
> > > > >> > My second reason is an optimistic appeal to a larger contributor
> > > base. I
> > > > >> > suspect that if we can modernize our infrastructure then we will
> > > > >> increase
> > > > >> > the pool of contributors who might be able to participate in
> this
> > > area.
> > > > >> I
> > > > >> > believe that GH Actions (and systems like it) is more prevalent
> in
> > > the
> > > > >> > industry than Jenkins, which means that more people already have
> > > > >> experience
> > > > >> > with the platform and more people will feel compelled to offer
> > > support
> > > > >> to
> > > > >> > an OSS project that uses the platform as a means of growing
> their
> > > own
> > > > >> > skillset and as a means of bolstering their CVs.
> > > > >> >
> > > > >> > Dove-tailed into reason two is reason three: I believe that
> there
> > > is a
> > > > >> > large community of folks who are developing GitHub Actions on
> its
> > > > >> > marketplace. We would effectively open ourselves up to more
> > > > >> off-the-shelf
> > > > >> > offerings and those offerings would be in our hands directly. By
> > > > >> contrast,
> > > > >> > I don't think there's as much development in Jenkins plugins,
> and
> > > the
> > > > >> > process of adding a new plugin to our Jenkins instance requires
> > > filing
> > > > >> an
> > > > >> > INFRA ticket.
> > > > >> >
> > > > >> > These are my motivations. I'm still not clear on what's possible
> > > yet for
> > > > >> > ASF projects. I have filed an INFRA ticket, requesting whatever
> is
> > > > >> > necessary for us to start an experiment. Indeed, I believe that
> > > there
> > > > >> are
> > > > >> > some major limitations on the current implementation provided
> by the
> > > > >> ASF,
> > > > >> > and as far as I can tell, only one project with a build
> footprint
> > > that
> > > > >> > resembles HBase has pursued this effort. I've catalogued the
> > > applicable
> > > > >> > information that I've found so far on that issue.
> > > > >> >
> > > > >> > https://issues.apache.org/jira/browse/INFRA-26170
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Nick
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Xiao Liu <li...@apache.org>.
Hi, Nick, got it, thank you for your patient explanation!!!
Best
On 2026/02/09 09:40:43 Nick Dimiduk wrote:
> Hi Xiao Liu,
>
> As far as I can tell, our Yetus invocations have always done that, both in
> Jenkins and GHA. I think it’s an artifact of the fact that the plugin was
> written against an earlier version of the GH permissions model. My
> understanding is that you no longer need any `repo:* write` permission to
> report the result of a check.
>
> Let take a note to get to the bottom of it.
>
> Thanks for taking a close look!
>
> -n
>
> On Mon, 9 Feb 2026 at 10:28, Xiao Liu <li...@apache.org> wrote:
>
> > Hi, Nick Dimiduk. Excellent work!
> >
> > I noticed that the current CI prints the error “ERROR: Failed to write
> > GitHub status. Token expired or missing repo:status write?” during the
> > Adding GitHub status phase(for example [1]). Is there an issue with this
> > error?
> >
> > [1]
> > https://github.com/apache/hbase/actions/runs/21818346708/job/62945085724?pr=7716
> >
> > Best
> >
> > On 2026/01/17 15:20:41 Nick Dimiduk wrote:
> > > TL;DR, we cannot have the full features of Jenkins-based checks via GHA
> > > because we cannot grant necessary permissions. This is a restriction on
> > any
> > > PR posted from an external repo. It’s not an ASF restriction but a GH
> > > restriction. I didn’t realize this initially because I did initial
> > > development from a branch off of the apache/hbase repo instead of my
> > > personal one.
> > >
> > > The only way our GHA pre-commit can interact with the PR seems to be via
> > > the Checks API. No more comment with summary of what was run. The action
> > > results are still available in full, I have a step that uploads them to
> > the
> > > GH archive system. But no longer are there convenient links off to the
> > > reports and logs.
> > >
> > > You can see an example of what all this looks like on
> > > https://github.com/apache/hbase/pull/7639 — look for activity from the
> > GHA
> > > robot, not from Jenkins (they have different names and icons).
> > >
> > > In my opinion, this is an acceptable compromise. I’d like to improve
> > Yetus
> > > so that every plugin that currently leaves a +1/-1 in the vote table also
> > > leaves a Check. Currently it seems only -1 results leave a Check.
> > >
> > > Any other suggestions or concerns?
> > >
> > > Thanks,
> > > Nick
> > >
> > > On Thu, 15 Jan 2026 at 15:50, Nick Dimiduk <nd...@apache.org> wrote:
> > >
> > > > I spoke a little too soon, there's some issue on non-master branches
> > that
> > > > didn't show on the back-port PRs. Filed what I know on INFRA-27571.
> > > >
> > > > -n
> > > >
> > > > On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org>
> > wrote:
> > > >
> > > >> Hi team,
> > > >>
> > > >> I've added GitHub Actions to run Yetus general checks directly on pull
> > > >> requests. This has been backported to all active branches (master,
> > > >> branch-3, branch-2, branch-2.6, branch-2.5).
> > > >>
> > > >> What's changing:
> > > >> - PRs will now trigger a "Yetus General Check" workflow that runs
> > > >> checkstyle, javac, pylint, shellcheck, and other static analysis
> > checks
> > > >> - Results appear as GitHub status checks and PR comments (same as
> > > >> Jenkins)
> > > >> - Old bot comments from previous runs are automatically hidden to
> > reduce
> > > >> noise
> > > >> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> > > >>
> > > >> What's NOT changing:
> > > >> - Jenkins precommit builds continue to run as before
> > > >> - Your PR workflow remains the same
> > > >> - Unit/integration tests still run on Jenkins, not GitHub Actions
> > > >>
> > > >> JIRAs: HBASE-29787, HBASE-29829
> > > >>
> > > >> Give us a shout if anything breaks for you. Also, feel free to pick up
> > > >> migrating other jobs, if you want to (I've not filed any subsequent
> > > >> tickets). It's a little bit tedious and it seems that you need push
> > rights
> > > >> to the origin github repo to introduce a new workflow (I'm not sure
> > about
> > > >> editing an existing workflow).
> > > >>
> > > >> Thanks,
> > > >> Nick
> > > >>
> > > >> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > > >> > Heya,
> > > >> >
> > > >> > I'd like to take the community temperature on migrating our build
> > infra
> > > >> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > > >> > Actions. I have several reasons that justify this proposal.
> > > >> >
> > > >> > As some of you may know, our community funding has reduced and we
> > will
> > > >> no
> > > >> > longer be able to sustain the current fleet of build
> > infrastructure. So,
> > > >> > one motivation for this proposal is cost-cutting: I think that
> > we'll be
> > > >> > able to operate at lower costs if we can migrate to a
> > > >> provisioned-as-needed
> > > >> > model of consumption.
> > > >> >
> > > >> > My second reason is an optimistic appeal to a larger contributor
> > base. I
> > > >> > suspect that if we can modernize our infrastructure then we will
> > > >> increase
> > > >> > the pool of contributors who might be able to participate in this
> > area.
> > > >> I
> > > >> > believe that GH Actions (and systems like it) is more prevalent in
> > the
> > > >> > industry than Jenkins, which means that more people already have
> > > >> experience
> > > >> > with the platform and more people will feel compelled to offer
> > support
> > > >> to
> > > >> > an OSS project that uses the platform as a means of growing their
> > own
> > > >> > skillset and as a means of bolstering their CVs.
> > > >> >
> > > >> > Dove-tailed into reason two is reason three: I believe that there
> > is a
> > > >> > large community of folks who are developing GitHub Actions on its
> > > >> > marketplace. We would effectively open ourselves up to more
> > > >> off-the-shelf
> > > >> > offerings and those offerings would be in our hands directly. By
> > > >> contrast,
> > > >> > I don't think there's as much development in Jenkins plugins, and
> > the
> > > >> > process of adding a new plugin to our Jenkins instance requires
> > filing
> > > >> an
> > > >> > INFRA ticket.
> > > >> >
> > > >> > These are my motivations. I'm still not clear on what's possible
> > yet for
> > > >> > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > > >> > necessary for us to start an experiment. Indeed, I believe that
> > there
> > > >> are
> > > >> > some major limitations on the current implementation provided by the
> > > >> ASF,
> > > >> > and as far as I can tell, only one project with a build footprint
> > that
> > > >> > resembles HBase has pursued this effort. I've catalogued the
> > applicable
> > > >> > information that I've found so far on that issue.
> > > >> >
> > > >> > https://issues.apache.org/jira/browse/INFRA-26170
> > > >> >
> > > >> > Thanks,
> > > >> > Nick
> > > >> >
> > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
Hi Xiao Liu,
As far as I can tell, our Yetus invocations have always done that, both in
Jenkins and GHA. I think it’s an artifact of the fact that the plugin was
written against an earlier version of the GH permissions model. My
understanding is that you no longer need any `repo:* write` permission to
report the result of a check.
Let take a note to get to the bottom of it.
Thanks for taking a close look!
-n
On Mon, 9 Feb 2026 at 10:28, Xiao Liu <li...@apache.org> wrote:
> Hi, Nick Dimiduk. Excellent work!
>
> I noticed that the current CI prints the error “ERROR: Failed to write
> GitHub status. Token expired or missing repo:status write?” during the
> Adding GitHub status phase(for example [1]). Is there an issue with this
> error?
>
> [1]
> https://github.com/apache/hbase/actions/runs/21818346708/job/62945085724?pr=7716
>
> Best
>
> On 2026/01/17 15:20:41 Nick Dimiduk wrote:
> > TL;DR, we cannot have the full features of Jenkins-based checks via GHA
> > because we cannot grant necessary permissions. This is a restriction on
> any
> > PR posted from an external repo. It’s not an ASF restriction but a GH
> > restriction. I didn’t realize this initially because I did initial
> > development from a branch off of the apache/hbase repo instead of my
> > personal one.
> >
> > The only way our GHA pre-commit can interact with the PR seems to be via
> > the Checks API. No more comment with summary of what was run. The action
> > results are still available in full, I have a step that uploads them to
> the
> > GH archive system. But no longer are there convenient links off to the
> > reports and logs.
> >
> > You can see an example of what all this looks like on
> > https://github.com/apache/hbase/pull/7639 — look for activity from the
> GHA
> > robot, not from Jenkins (they have different names and icons).
> >
> > In my opinion, this is an acceptable compromise. I’d like to improve
> Yetus
> > so that every plugin that currently leaves a +1/-1 in the vote table also
> > leaves a Check. Currently it seems only -1 results leave a Check.
> >
> > Any other suggestions or concerns?
> >
> > Thanks,
> > Nick
> >
> > On Thu, 15 Jan 2026 at 15:50, Nick Dimiduk <nd...@apache.org> wrote:
> >
> > > I spoke a little too soon, there's some issue on non-master branches
> that
> > > didn't show on the back-port PRs. Filed what I know on INFRA-27571.
> > >
> > > -n
> > >
> > > On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org>
> wrote:
> > >
> > >> Hi team,
> > >>
> > >> I've added GitHub Actions to run Yetus general checks directly on pull
> > >> requests. This has been backported to all active branches (master,
> > >> branch-3, branch-2, branch-2.6, branch-2.5).
> > >>
> > >> What's changing:
> > >> - PRs will now trigger a "Yetus General Check" workflow that runs
> > >> checkstyle, javac, pylint, shellcheck, and other static analysis
> checks
> > >> - Results appear as GitHub status checks and PR comments (same as
> > >> Jenkins)
> > >> - Old bot comments from previous runs are automatically hidden to
> reduce
> > >> noise
> > >> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> > >>
> > >> What's NOT changing:
> > >> - Jenkins precommit builds continue to run as before
> > >> - Your PR workflow remains the same
> > >> - Unit/integration tests still run on Jenkins, not GitHub Actions
> > >>
> > >> JIRAs: HBASE-29787, HBASE-29829
> > >>
> > >> Give us a shout if anything breaks for you. Also, feel free to pick up
> > >> migrating other jobs, if you want to (I've not filed any subsequent
> > >> tickets). It's a little bit tedious and it seems that you need push
> rights
> > >> to the origin github repo to introduce a new workflow (I'm not sure
> about
> > >> editing an existing workflow).
> > >>
> > >> Thanks,
> > >> Nick
> > >>
> > >> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > >> > Heya,
> > >> >
> > >> > I'd like to take the community temperature on migrating our build
> infra
> > >> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > >> > Actions. I have several reasons that justify this proposal.
> > >> >
> > >> > As some of you may know, our community funding has reduced and we
> will
> > >> no
> > >> > longer be able to sustain the current fleet of build
> infrastructure. So,
> > >> > one motivation for this proposal is cost-cutting: I think that
> we'll be
> > >> > able to operate at lower costs if we can migrate to a
> > >> provisioned-as-needed
> > >> > model of consumption.
> > >> >
> > >> > My second reason is an optimistic appeal to a larger contributor
> base. I
> > >> > suspect that if we can modernize our infrastructure then we will
> > >> increase
> > >> > the pool of contributors who might be able to participate in this
> area.
> > >> I
> > >> > believe that GH Actions (and systems like it) is more prevalent in
> the
> > >> > industry than Jenkins, which means that more people already have
> > >> experience
> > >> > with the platform and more people will feel compelled to offer
> support
> > >> to
> > >> > an OSS project that uses the platform as a means of growing their
> own
> > >> > skillset and as a means of bolstering their CVs.
> > >> >
> > >> > Dove-tailed into reason two is reason three: I believe that there
> is a
> > >> > large community of folks who are developing GitHub Actions on its
> > >> > marketplace. We would effectively open ourselves up to more
> > >> off-the-shelf
> > >> > offerings and those offerings would be in our hands directly. By
> > >> contrast,
> > >> > I don't think there's as much development in Jenkins plugins, and
> the
> > >> > process of adding a new plugin to our Jenkins instance requires
> filing
> > >> an
> > >> > INFRA ticket.
> > >> >
> > >> > These are my motivations. I'm still not clear on what's possible
> yet for
> > >> > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > >> > necessary for us to start an experiment. Indeed, I believe that
> there
> > >> are
> > >> > some major limitations on the current implementation provided by the
> > >> ASF,
> > >> > and as far as I can tell, only one project with a build footprint
> that
> > >> > resembles HBase has pursued this effort. I've catalogued the
> applicable
> > >> > information that I've found so far on that issue.
> > >> >
> > >> > https://issues.apache.org/jira/browse/INFRA-26170
> > >> >
> > >> > Thanks,
> > >> > Nick
> > >> >
> > >>
> > >
> >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Xiao Liu <li...@apache.org>.
Hi, Nick Dimiduk. Excellent work!
I noticed that the current CI prints the error “ERROR: Failed to write GitHub status. Token expired or missing repo:status write?” during the Adding GitHub status phase(for example [1]). Is there an issue with this error?
[1] https://github.com/apache/hbase/actions/runs/21818346708/job/62945085724?pr=7716
Best
On 2026/01/17 15:20:41 Nick Dimiduk wrote:
> TL;DR, we cannot have the full features of Jenkins-based checks via GHA
> because we cannot grant necessary permissions. This is a restriction on any
> PR posted from an external repo. It’s not an ASF restriction but a GH
> restriction. I didn’t realize this initially because I did initial
> development from a branch off of the apache/hbase repo instead of my
> personal one.
>
> The only way our GHA pre-commit can interact with the PR seems to be via
> the Checks API. No more comment with summary of what was run. The action
> results are still available in full, I have a step that uploads them to the
> GH archive system. But no longer are there convenient links off to the
> reports and logs.
>
> You can see an example of what all this looks like on
> https://github.com/apache/hbase/pull/7639 — look for activity from the GHA
> robot, not from Jenkins (they have different names and icons).
>
> In my opinion, this is an acceptable compromise. I’d like to improve Yetus
> so that every plugin that currently leaves a +1/-1 in the vote table also
> leaves a Check. Currently it seems only -1 results leave a Check.
>
> Any other suggestions or concerns?
>
> Thanks,
> Nick
>
> On Thu, 15 Jan 2026 at 15:50, Nick Dimiduk <nd...@apache.org> wrote:
>
> > I spoke a little too soon, there's some issue on non-master branches that
> > didn't show on the back-port PRs. Filed what I know on INFRA-27571.
> >
> > -n
> >
> > On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org> wrote:
> >
> >> Hi team,
> >>
> >> I've added GitHub Actions to run Yetus general checks directly on pull
> >> requests. This has been backported to all active branches (master,
> >> branch-3, branch-2, branch-2.6, branch-2.5).
> >>
> >> What's changing:
> >> - PRs will now trigger a "Yetus General Check" workflow that runs
> >> checkstyle, javac, pylint, shellcheck, and other static analysis checks
> >> - Results appear as GitHub status checks and PR comments (same as
> >> Jenkins)
> >> - Old bot comments from previous runs are automatically hidden to reduce
> >> noise
> >> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
> >>
> >> What's NOT changing:
> >> - Jenkins precommit builds continue to run as before
> >> - Your PR workflow remains the same
> >> - Unit/integration tests still run on Jenkins, not GitHub Actions
> >>
> >> JIRAs: HBASE-29787, HBASE-29829
> >>
> >> Give us a shout if anything breaks for you. Also, feel free to pick up
> >> migrating other jobs, if you want to (I've not filed any subsequent
> >> tickets). It's a little bit tedious and it seems that you need push rights
> >> to the origin github repo to introduce a new workflow (I'm not sure about
> >> editing an existing workflow).
> >>
> >> Thanks,
> >> Nick
> >>
> >> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> >> > Heya,
> >> >
> >> > I'd like to take the community temperature on migrating our build infra
> >> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> >> > Actions. I have several reasons that justify this proposal.
> >> >
> >> > As some of you may know, our community funding has reduced and we will
> >> no
> >> > longer be able to sustain the current fleet of build infrastructure. So,
> >> > one motivation for this proposal is cost-cutting: I think that we'll be
> >> > able to operate at lower costs if we can migrate to a
> >> provisioned-as-needed
> >> > model of consumption.
> >> >
> >> > My second reason is an optimistic appeal to a larger contributor base. I
> >> > suspect that if we can modernize our infrastructure then we will
> >> increase
> >> > the pool of contributors who might be able to participate in this area.
> >> I
> >> > believe that GH Actions (and systems like it) is more prevalent in the
> >> > industry than Jenkins, which means that more people already have
> >> experience
> >> > with the platform and more people will feel compelled to offer support
> >> to
> >> > an OSS project that uses the platform as a means of growing their own
> >> > skillset and as a means of bolstering their CVs.
> >> >
> >> > Dove-tailed into reason two is reason three: I believe that there is a
> >> > large community of folks who are developing GitHub Actions on its
> >> > marketplace. We would effectively open ourselves up to more
> >> off-the-shelf
> >> > offerings and those offerings would be in our hands directly. By
> >> contrast,
> >> > I don't think there's as much development in Jenkins plugins, and the
> >> > process of adding a new plugin to our Jenkins instance requires filing
> >> an
> >> > INFRA ticket.
> >> >
> >> > These are my motivations. I'm still not clear on what's possible yet for
> >> > ASF projects. I have filed an INFRA ticket, requesting whatever is
> >> > necessary for us to start an experiment. Indeed, I believe that there
> >> are
> >> > some major limitations on the current implementation provided by the
> >> ASF,
> >> > and as far as I can tell, only one project with a build footprint that
> >> > resembles HBase has pursued this effort. I've catalogued the applicable
> >> > information that I've found so far on that issue.
> >> >
> >> > https://issues.apache.org/jira/browse/INFRA-26170
> >> >
> >> > Thanks,
> >> > Nick
> >> >
> >>
> >
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
TL;DR, we cannot have the full features of Jenkins-based checks via GHA
because we cannot grant necessary permissions. This is a restriction on any
PR posted from an external repo. It’s not an ASF restriction but a GH
restriction. I didn’t realize this initially because I did initial
development from a branch off of the apache/hbase repo instead of my
personal one.
The only way our GHA pre-commit can interact with the PR seems to be via
the Checks API. No more comment with summary of what was run. The action
results are still available in full, I have a step that uploads them to the
GH archive system. But no longer are there convenient links off to the
reports and logs.
You can see an example of what all this looks like on
https://github.com/apache/hbase/pull/7639 — look for activity from the GHA
robot, not from Jenkins (they have different names and icons).
In my opinion, this is an acceptable compromise. I’d like to improve Yetus
so that every plugin that currently leaves a +1/-1 in the vote table also
leaves a Check. Currently it seems only -1 results leave a Check.
Any other suggestions or concerns?
Thanks,
Nick
On Thu, 15 Jan 2026 at 15:50, Nick Dimiduk <nd...@apache.org> wrote:
> I spoke a little too soon, there's some issue on non-master branches that
> didn't show on the back-port PRs. Filed what I know on INFRA-27571.
>
> -n
>
> On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org> wrote:
>
>> Hi team,
>>
>> I've added GitHub Actions to run Yetus general checks directly on pull
>> requests. This has been backported to all active branches (master,
>> branch-3, branch-2, branch-2.6, branch-2.5).
>>
>> What's changing:
>> - PRs will now trigger a "Yetus General Check" workflow that runs
>> checkstyle, javac, pylint, shellcheck, and other static analysis checks
>> - Results appear as GitHub status checks and PR comments (same as
>> Jenkins)
>> - Old bot comments from previous runs are automatically hidden to reduce
>> noise
>> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
>>
>> What's NOT changing:
>> - Jenkins precommit builds continue to run as before
>> - Your PR workflow remains the same
>> - Unit/integration tests still run on Jenkins, not GitHub Actions
>>
>> JIRAs: HBASE-29787, HBASE-29829
>>
>> Give us a shout if anything breaks for you. Also, feel free to pick up
>> migrating other jobs, if you want to (I've not filed any subsequent
>> tickets). It's a little bit tedious and it seems that you need push rights
>> to the origin github repo to introduce a new workflow (I'm not sure about
>> editing an existing workflow).
>>
>> Thanks,
>> Nick
>>
>> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
>> > Heya,
>> >
>> > I'd like to take the community temperature on migrating our build infra
>> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
>> > Actions. I have several reasons that justify this proposal.
>> >
>> > As some of you may know, our community funding has reduced and we will
>> no
>> > longer be able to sustain the current fleet of build infrastructure. So,
>> > one motivation for this proposal is cost-cutting: I think that we'll be
>> > able to operate at lower costs if we can migrate to a
>> provisioned-as-needed
>> > model of consumption.
>> >
>> > My second reason is an optimistic appeal to a larger contributor base. I
>> > suspect that if we can modernize our infrastructure then we will
>> increase
>> > the pool of contributors who might be able to participate in this area.
>> I
>> > believe that GH Actions (and systems like it) is more prevalent in the
>> > industry than Jenkins, which means that more people already have
>> experience
>> > with the platform and more people will feel compelled to offer support
>> to
>> > an OSS project that uses the platform as a means of growing their own
>> > skillset and as a means of bolstering their CVs.
>> >
>> > Dove-tailed into reason two is reason three: I believe that there is a
>> > large community of folks who are developing GitHub Actions on its
>> > marketplace. We would effectively open ourselves up to more
>> off-the-shelf
>> > offerings and those offerings would be in our hands directly. By
>> contrast,
>> > I don't think there's as much development in Jenkins plugins, and the
>> > process of adding a new plugin to our Jenkins instance requires filing
>> an
>> > INFRA ticket.
>> >
>> > These are my motivations. I'm still not clear on what's possible yet for
>> > ASF projects. I have filed an INFRA ticket, requesting whatever is
>> > necessary for us to start an experiment. Indeed, I believe that there
>> are
>> > some major limitations on the current implementation provided by the
>> ASF,
>> > and as far as I can tell, only one project with a build footprint that
>> > resembles HBase has pursued this effort. I've catalogued the applicable
>> > information that I've found so far on that issue.
>> >
>> > https://issues.apache.org/jira/browse/INFRA-26170
>> >
>> > Thanks,
>> > Nick
>> >
>>
>
Re: [DISCUSS] Migrate from Jenkins to GitHub Actions for CI
Posted by Nick Dimiduk <nd...@apache.org>.
I spoke a little too soon, there's some issue on non-master branches that
didn't show on the back-port PRs. Filed what I know on INFRA-27571.
-n
On Thu, Jan 15, 2026 at 3:37 PM Nick Dimiduk <nd...@apache.org> wrote:
> Hi team,
>
> I've added GitHub Actions to run Yetus general checks directly on pull
> requests. This has been backported to all active branches (master,
> branch-3, branch-2, branch-2.6, branch-2.5).
>
> What's changing:
> - PRs will now trigger a "Yetus General Check" workflow that runs
> checkstyle, javac, pylint, shellcheck, and other static analysis checks
> - Results appear as GitHub status checks and PR comments (same as Jenkins)
> - Old bot comments from previous runs are automatically hidden to reduce
> noise
> - Uses the same scripts, Dockerfile, &c. that we use for Jenkins
>
> What's NOT changing:
> - Jenkins precommit builds continue to run as before
> - Your PR workflow remains the same
> - Unit/integration tests still run on Jenkins, not GitHub Actions
>
> JIRAs: HBASE-29787, HBASE-29829
>
> Give us a shout if anything breaks for you. Also, feel free to pick up
> migrating other jobs, if you want to (I've not filed any subsequent
> tickets). It's a little bit tedious and it seems that you need push rights
> to the origin github repo to introduce a new workflow (I'm not sure about
> editing an existing workflow).
>
> Thanks,
> Nick
>
> On 2024/10/02 09:22:51 Nick Dimiduk wrote:
> > Heya,
> >
> > I'd like to take the community temperature on migrating our build infra
> > from the ci-hbase.a.o Jenkins instance to something built on GitHub
> > Actions. I have several reasons that justify this proposal.
> >
> > As some of you may know, our community funding has reduced and we will no
> > longer be able to sustain the current fleet of build infrastructure. So,
> > one motivation for this proposal is cost-cutting: I think that we'll be
> > able to operate at lower costs if we can migrate to a
> provisioned-as-needed
> > model of consumption.
> >
> > My second reason is an optimistic appeal to a larger contributor base. I
> > suspect that if we can modernize our infrastructure then we will increase
> > the pool of contributors who might be able to participate in this area. I
> > believe that GH Actions (and systems like it) is more prevalent in the
> > industry than Jenkins, which means that more people already have
> experience
> > with the platform and more people will feel compelled to offer support to
> > an OSS project that uses the platform as a means of growing their own
> > skillset and as a means of bolstering their CVs.
> >
> > Dove-tailed into reason two is reason three: I believe that there is a
> > large community of folks who are developing GitHub Actions on its
> > marketplace. We would effectively open ourselves up to more off-the-shelf
> > offerings and those offerings would be in our hands directly. By
> contrast,
> > I don't think there's as much development in Jenkins plugins, and the
> > process of adding a new plugin to our Jenkins instance requires filing an
> > INFRA ticket.
> >
> > These are my motivations. I'm still not clear on what's possible yet for
> > ASF projects. I have filed an INFRA ticket, requesting whatever is
> > necessary for us to start an experiment. Indeed, I believe that there are
> > some major limitations on the current implementation provided by the ASF,
> > and as far as I can tell, only one project with a build footprint that
> > resembles HBase has pursued this effort. I've catalogued the applicable
> > information that I've found so far on that issue.
> >
> > https://issues.apache.org/jira/browse/INFRA-26170
> >
> > Thanks,
> > Nick
> >
>