You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Krisztián Szűcs <sz...@gmail.com> on 2021/01/05 12:33:38 UTC
Github Actions feedback time
Hi,
I'm concerned about the overall feedback time we have on pull requests.
I have a simple PR to make the comment bot working again, but no
builds are running even after 30 minutes.
I can see 2-4 running builds, which will make our work much harder
right before the release.
I wasn't following the build queue's state lately, but I think we
should consolidate the build configurations.
Possible candidates are the PR* workflows and good to have tests which
we could trigger on master instead.
Opinions?
Regards, Krisztian
Re: Github Actions feedback time
Posted by Ying Zhou <yz...@gmail.com>.
Hi,
Sorry for not noticing this thread earlier. Looks like in addition to unusually slow feedback time that did not happen last Sunday or earlier there are also weird installation errors such as ‘can not install numpy’ as well. Can these be due to some form of timeout?
Here is my C++ PR:
https://github.com/apache/arrow/pull/8648/checks
Ying
> On Jan 5, 2021, at 7:33 AM, Krisztián Szűcs <sz...@gmail.com> wrote:
>
> Hi,
>
> I'm concerned about the overall feedback time we have on pull requests.
> I have a simple PR to make the comment bot working again, but no
> builds are running even after 30 minutes.
> I can see 2-4 running builds, which will make our work much harder
> right before the release.
>
> I wasn't following the build queue's state lately, but I think we
> should consolidate the build configurations.
> Possible candidates are the PR* workflows and good to have tests which
> we could trigger on master instead.
>
> Opinions?
>
> Regards, Krisztian
Re: Github Actions feedback time
Posted by Krisztián Szűcs <sz...@gmail.com>.
On Tue, Jan 5, 2021 at 3:21 PM Jorge Cardoso Leitão
<jo...@gmail.com> wrote:
>
> Hi Krisztian,
>
> Could you describe which flows you are referring to?
>
> Rust builds are taking 4-6m on non-(windows/mac) and 8-15m on windows/mac,
> with almost immediate feedback. E.g. I force-pushed recently
> https://github.com/apache/arrow/pull/9089 and the first complete test came
> 5m later.
On the actions page I see builds with 30m queue time:
https://github.com/apache/arrow/actions
>
> One hypothesis is that, on personal repos, many builds are being triggered
> when any change happens. On my repo, R
> <https://github.com/jorgecarleitao/arrow/actions/runs/463436428>, Python
> <https://github.com/jorgecarleitao/arrow/actions/runs/463436416>, C++
> <https://github.com/jorgecarleitao/arrow/actions/runs/463436422>, Java JNI
> <https://github.com/jorgecarleitao/arrow/actions/runs/463436419>, C GLib
> and Ruby <https://github.com/jorgecarleitao/arrow/actions/runs/463436420>
> jobs ran when a Rust-only change was force pushed into a branch.
I'm not sure what could we do against it, but it could worth some investigation.
>
> Another hypothesis is that there are too many builds being triggered with a
> blanket `ci/*`, which covers all languages, and thus, when changing the
> comment bot, they all get triggered.
I think we improved that, see the cpp paths as an example:
https://github.com/apache/arrow/blob/master/.github/workflows/cpp.yml#L22
>
> What I usually do when playing with builds that can't be tested on my
> personal repo (e.g. because they require credentials), is to delete all
> unrelated workflows during testing, and then checkout them when the PR is
> ready. A bit hackish I must say, but works.
To prevent triggering the upstream builds you can add `WIP` to the PR's title:
https://github.com/apache/arrow/blob/master/.github/workflows/cpp.yml#L52
>
> Best,
> Jorge
>
>
> On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs <sz...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm concerned about the overall feedback time we have on pull requests.
> > I have a simple PR to make the comment bot working again, but no
> > builds are running even after 30 minutes.
> > I can see 2-4 running builds, which will make our work much harder
> > right before the release.
> >
> > I wasn't following the build queue's state lately, but I think we
> > should consolidate the build configurations.
> > Possible candidates are the PR* workflows and good to have tests which
> > we could trigger on master instead.
> >
> > Opinions?
> >
> > Regards, Krisztian
> >
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi Krisztian,
Could you describe which flows you are referring to?
Rust builds are taking 4-6m on non-(windows/mac) and 8-15m on windows/mac,
with almost immediate feedback. E.g. I force-pushed recently
https://github.com/apache/arrow/pull/9089 and the first complete test came
5m later.
One hypothesis is that, on personal repos, many builds are being triggered
when any change happens. On my repo, R
<https://github.com/jorgecarleitao/arrow/actions/runs/463436428>, Python
<https://github.com/jorgecarleitao/arrow/actions/runs/463436416>, C++
<https://github.com/jorgecarleitao/arrow/actions/runs/463436422>, Java JNI
<https://github.com/jorgecarleitao/arrow/actions/runs/463436419>, C GLib
and Ruby <https://github.com/jorgecarleitao/arrow/actions/runs/463436420>
jobs ran when a Rust-only change was force pushed into a branch.
Another hypothesis is that there are too many builds being triggered with a
blanket `ci/*`, which covers all languages, and thus, when changing the
comment bot, they all get triggered.
What I usually do when playing with builds that can't be tested on my
personal repo (e.g. because they require credentials), is to delete all
unrelated workflows during testing, and then checkout them when the PR is
ready. A bit hackish I must say, but works.
Best,
Jorge
On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs <sz...@gmail.com>
wrote:
> Hi,
>
> I'm concerned about the overall feedback time we have on pull requests.
> I have a simple PR to make the comment bot working again, but no
> builds are running even after 30 minutes.
> I can see 2-4 running builds, which will make our work much harder
> right before the release.
>
> I wasn't following the build queue's state lately, but I think we
> should consolidate the build configurations.
> Possible candidates are the PR* workflows and good to have tests which
> we could trigger on master instead.
>
> Opinions?
>
> Regards, Krisztian
>
Re: Github Actions feedback time
Posted by Josh Taylor <jo...@gmail.com>.
Random question - could Github be reached out to for the limits to be
increased/investigated in the mean time, given this is an
opensource project?
On Wed, 6 Jan 2021 at 06:57, Andrew Lamb <al...@influxdata.com> wrote:
> Makes sense -- thank you
>
> On Tue, Jan 5, 2021 at 5:38 PM Wes McKinney <we...@gmail.com> wrote:
>
> > There are a lot of companies now with a business interest in the success
> of
> > Apache Arrow. We should do a back of the envelope estimate of the number
> of
> > compute hours we use per year and arrive at an annual cost estimate. My
> > guess (and maybe I’m wildly off) is that it’s somewhere around
> > $50,000-100,000/year, which shouldn’t be difficult to come by. Costs
> could
> > also be mitigated by organizations hooking up physical on-prem build
> > machines.
> >
> > On Tue, Jan 5, 2021 at 4:29 PM Andrew Lamb <al...@influxdata.com> wrote:
> >
> > > One thing that is not clear to me is who will pay for the dedicated
> > > infrastructure costs with something like Build Kite?
> > >
> > > I don't fully understand how moving to self hosted (and thus also
> > > presumably limited) arrow-only infrastructure will reduce build queue
> > > times, unless there is a deep pocketed sponsor. I can certainly see how
> > > moving to self hosted infrastructure will increase the amount of
> control
> > /
> > > visibility the project would have
> > >
> > > Andrew
> > >
> > >
> > >
> > > On Tue, Jan 5, 2021 at 12:46 PM Jorge Cardoso Leitão <
> > > jorgecarleitao@gmail.com> wrote:
> > >
> > > > Krisztián,
> > > >
> > > > I agree with you that there is an ongoing problem with the queue.
> > Thanks
> > > > for the tips wrt to WIP.
> > > >
> > > > Wes, I would be up for moving Rust workflows to Buildkite. Is the
> > > > integration in place wrt to reporting to triggers and reporting back
> to
> > > > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is
> > there
> > > > any PMC-specific activity that is blocking us from working on this?
> > > >
> > > > Best,
> > > > Jorge
> > > >
> > > >
> > > >
> > > >
> > > > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com>
> > wrote:
> > > >
> > > > > At the risk of sounding like a broken record -- we are almost
> > > > > certainly going to have to move our builds to dedicated
> > infrastructure
> > > > > that this community has complete agency over sometime between now
> and
> > > > > 2025. Maybe it will be this year, maybe next year, but to me it is
> an
> > > > > inevitability. I spent quite a bit of time getting Apache Infra to
> > > > > enable Buildkite for apache/* repositories precisely so that we
> have
> > a
> > > > > coherent path. I hope to be able to have folks on my team focus on
> > > > > this at some point and contribute it to the community once we have
> > the
> > > > > bandwidth.
> > > > >
> > > > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > > > > <sz...@gmail.com> wrote:
> > > > > >
> > > > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <jeroenooms@gmail.com
> >
> > > > wrote:
> > > > > > >
> > > > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > > > > <sz...@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I'm concerned about the overall feedback time we have on pull
> > > > > requests.
> > > > > > > > I have a simple PR to make the comment bot working again, but
> > no
> > > > > > > > builds are running even after 30 minutes.
> > > > > > >
> > > > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have
> > the
> > > > > > > same problem sometimes with pull requests taking long to build,
> > but
> > > > > > > this usually happens when there are large jobs running under my
> > > name
> > > > > > > in completely unrelated projects. Also note that if you use
> e.g.
> > > > > > > scheduled (cron) actions, then the jobs are executed in the
> queue
> > > of
> > > > > > > the github user who last touched the yml file for that
> workflow.
> > > > > > Based on the "Actions" tab the queueing seems chronological to
> me,
> > > > > > although I can imagine the behaviour you described.
> > > > > >
> > > > > > Has anyone experienced similar clogging on apache/arrow's gha
> queue
> > > > > lately?
> > > > >
> > > >
> > >
> >
>
Re: Github Actions feedback time
Posted by Andrew Lamb <al...@influxdata.com>.
Makes sense -- thank you
On Tue, Jan 5, 2021 at 5:38 PM Wes McKinney <we...@gmail.com> wrote:
> There are a lot of companies now with a business interest in the success of
> Apache Arrow. We should do a back of the envelope estimate of the number of
> compute hours we use per year and arrive at an annual cost estimate. My
> guess (and maybe I’m wildly off) is that it’s somewhere around
> $50,000-100,000/year, which shouldn’t be difficult to come by. Costs could
> also be mitigated by organizations hooking up physical on-prem build
> machines.
>
> On Tue, Jan 5, 2021 at 4:29 PM Andrew Lamb <al...@influxdata.com> wrote:
>
> > One thing that is not clear to me is who will pay for the dedicated
> > infrastructure costs with something like Build Kite?
> >
> > I don't fully understand how moving to self hosted (and thus also
> > presumably limited) arrow-only infrastructure will reduce build queue
> > times, unless there is a deep pocketed sponsor. I can certainly see how
> > moving to self hosted infrastructure will increase the amount of control
> /
> > visibility the project would have
> >
> > Andrew
> >
> >
> >
> > On Tue, Jan 5, 2021 at 12:46 PM Jorge Cardoso Leitão <
> > jorgecarleitao@gmail.com> wrote:
> >
> > > Krisztián,
> > >
> > > I agree with you that there is an ongoing problem with the queue.
> Thanks
> > > for the tips wrt to WIP.
> > >
> > > Wes, I would be up for moving Rust workflows to Buildkite. Is the
> > > integration in place wrt to reporting to triggers and reporting back to
> > > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is
> there
> > > any PMC-specific activity that is blocking us from working on this?
> > >
> > > Best,
> > > Jorge
> > >
> > >
> > >
> > >
> > > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com>
> wrote:
> > >
> > > > At the risk of sounding like a broken record -- we are almost
> > > > certainly going to have to move our builds to dedicated
> infrastructure
> > > > that this community has complete agency over sometime between now and
> > > > 2025. Maybe it will be this year, maybe next year, but to me it is an
> > > > inevitability. I spent quite a bit of time getting Apache Infra to
> > > > enable Buildkite for apache/* repositories precisely so that we have
> a
> > > > coherent path. I hope to be able to have folks on my team focus on
> > > > this at some point and contribute it to the community once we have
> the
> > > > bandwidth.
> > > >
> > > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > > > <sz...@gmail.com> wrote:
> > > > >
> > > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > > > <sz...@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'm concerned about the overall feedback time we have on pull
> > > > requests.
> > > > > > > I have a simple PR to make the comment bot working again, but
> no
> > > > > > > builds are running even after 30 minutes.
> > > > > >
> > > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have
> the
> > > > > > same problem sometimes with pull requests taking long to build,
> but
> > > > > > this usually happens when there are large jobs running under my
> > name
> > > > > > in completely unrelated projects. Also note that if you use e.g.
> > > > > > scheduled (cron) actions, then the jobs are executed in the queue
> > of
> > > > > > the github user who last touched the yml file for that workflow.
> > > > > Based on the "Actions" tab the queueing seems chronological to me,
> > > > > although I can imagine the behaviour you described.
> > > > >
> > > > > Has anyone experienced similar clogging on apache/arrow's gha queue
> > > > lately?
> > > >
> > >
> >
>
Re: Github Actions feedback time
Posted by Wes McKinney <we...@gmail.com>.
There are a lot of companies now with a business interest in the success of
Apache Arrow. We should do a back of the envelope estimate of the number of
compute hours we use per year and arrive at an annual cost estimate. My
guess (and maybe I’m wildly off) is that it’s somewhere around
$50,000-100,000/year, which shouldn’t be difficult to come by. Costs could
also be mitigated by organizations hooking up physical on-prem build
machines.
On Tue, Jan 5, 2021 at 4:29 PM Andrew Lamb <al...@influxdata.com> wrote:
> One thing that is not clear to me is who will pay for the dedicated
> infrastructure costs with something like Build Kite?
>
> I don't fully understand how moving to self hosted (and thus also
> presumably limited) arrow-only infrastructure will reduce build queue
> times, unless there is a deep pocketed sponsor. I can certainly see how
> moving to self hosted infrastructure will increase the amount of control /
> visibility the project would have
>
> Andrew
>
>
>
> On Tue, Jan 5, 2021 at 12:46 PM Jorge Cardoso Leitão <
> jorgecarleitao@gmail.com> wrote:
>
> > Krisztián,
> >
> > I agree with you that there is an ongoing problem with the queue. Thanks
> > for the tips wrt to WIP.
> >
> > Wes, I would be up for moving Rust workflows to Buildkite. Is the
> > integration in place wrt to reporting to triggers and reporting back to
> > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there
> > any PMC-specific activity that is blocking us from working on this?
> >
> > Best,
> > Jorge
> >
> >
> >
> >
> > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > > At the risk of sounding like a broken record -- we are almost
> > > certainly going to have to move our builds to dedicated infrastructure
> > > that this community has complete agency over sometime between now and
> > > 2025. Maybe it will be this year, maybe next year, but to me it is an
> > > inevitability. I spent quite a bit of time getting Apache Infra to
> > > enable Buildkite for apache/* repositories precisely so that we have a
> > > coherent path. I hope to be able to have folks on my team focus on
> > > this at some point and contribute it to the community once we have the
> > > bandwidth.
> > >
> > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > > <sz...@gmail.com> wrote:
> > > >
> > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com>
> > wrote:
> > > > >
> > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > > <sz...@gmail.com> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm concerned about the overall feedback time we have on pull
> > > requests.
> > > > > > I have a simple PR to make the comment bot working again, but no
> > > > > > builds are running even after 30 minutes.
> > > > >
> > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > > > > same problem sometimes with pull requests taking long to build, but
> > > > > this usually happens when there are large jobs running under my
> name
> > > > > in completely unrelated projects. Also note that if you use e.g.
> > > > > scheduled (cron) actions, then the jobs are executed in the queue
> of
> > > > > the github user who last touched the yml file for that workflow.
> > > > Based on the "Actions" tab the queueing seems chronological to me,
> > > > although I can imagine the behaviour you described.
> > > >
> > > > Has anyone experienced similar clogging on apache/arrow's gha queue
> > > lately?
> > >
> >
>
Re: Github Actions feedback time
Posted by Andrew Lamb <al...@influxdata.com>.
One thing that is not clear to me is who will pay for the dedicated
infrastructure costs with something like Build Kite?
I don't fully understand how moving to self hosted (and thus also
presumably limited) arrow-only infrastructure will reduce build queue
times, unless there is a deep pocketed sponsor. I can certainly see how
moving to self hosted infrastructure will increase the amount of control /
visibility the project would have
Andrew
On Tue, Jan 5, 2021 at 12:46 PM Jorge Cardoso Leitão <
jorgecarleitao@gmail.com> wrote:
> Krisztián,
>
> I agree with you that there is an ongoing problem with the queue. Thanks
> for the tips wrt to WIP.
>
> Wes, I would be up for moving Rust workflows to Buildkite. Is the
> integration in place wrt to reporting to triggers and reporting back to
> gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there
> any PMC-specific activity that is blocking us from working on this?
>
> Best,
> Jorge
>
>
>
>
> On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com> wrote:
>
> > At the risk of sounding like a broken record -- we are almost
> > certainly going to have to move our builds to dedicated infrastructure
> > that this community has complete agency over sometime between now and
> > 2025. Maybe it will be this year, maybe next year, but to me it is an
> > inevitability. I spent quite a bit of time getting Apache Infra to
> > enable Buildkite for apache/* repositories precisely so that we have a
> > coherent path. I hope to be able to have folks on my team focus on
> > this at some point and contribute it to the community once we have the
> > bandwidth.
> >
> > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > <sz...@gmail.com> wrote:
> > >
> > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com>
> wrote:
> > > >
> > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > <sz...@gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'm concerned about the overall feedback time we have on pull
> > requests.
> > > > > I have a simple PR to make the comment bot working again, but no
> > > > > builds are running even after 30 minutes.
> > > >
> > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > > > same problem sometimes with pull requests taking long to build, but
> > > > this usually happens when there are large jobs running under my name
> > > > in completely unrelated projects. Also note that if you use e.g.
> > > > scheduled (cron) actions, then the jobs are executed in the queue of
> > > > the github user who last touched the yml file for that workflow.
> > > Based on the "Actions" tab the queueing seems chronological to me,
> > > although I can imagine the behaviour you described.
> > >
> > > Has anyone experienced similar clogging on apache/arrow's gha queue
> > lately?
> >
>
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi Wes,
I fielded ARROW-11140 <https://issues.apache.org/jira/browse/ARROW-11140>
with what I would consider a successful POC from Rust implementation's
point of view. Let's move the discussion there.
I was already able to run Rust's first steps there (using my own mac as a
runner), and, so far, looks pretty good I must say. Would it be possible to
add a linux agent (amd64) on it? I would expect
https://github.com/apache/arrow/pull/9111 to be triggered.
Best,
Jorge
On Tue, Jan 5, 2021 at 11:33 PM Wes McKinney <we...@gmail.com> wrote:
> hi Jorge -- as far as I know, all that's necessary to use BK is the
> pipeline configuration in the repo and to attach workers to Arrow's
> (free) account (which I'm happy to add you to). We can likely get a
> lot of free cloud credits from Google, Microsoft, or Amazon if we
> wanted to go that route to keep costs down (I could help pay for the
> cloud bill, too).
>
> The one thing to be aware of with BK is the possible security risks
> from builds running on pull requests. With reasonable security
> practices (not putting sensitive data on the build hosts) this should
> be too much of a problem.
>
> On Tue, Jan 5, 2021 at 11:46 AM Jorge Cardoso Leitão
> <jo...@gmail.com> wrote:
> >
> > Krisztián,
> >
> > I agree with you that there is an ongoing problem with the queue. Thanks
> > for the tips wrt to WIP.
> >
> > Wes, I would be up for moving Rust workflows to Buildkite. Is the
> > integration in place wrt to reporting to triggers and reporting back to
> > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there
> > any PMC-specific activity that is blocking us from working on this?
> >
> > Best,
> > Jorge
> >
> >
> >
> >
> > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > > At the risk of sounding like a broken record -- we are almost
> > > certainly going to have to move our builds to dedicated infrastructure
> > > that this community has complete agency over sometime between now and
> > > 2025. Maybe it will be this year, maybe next year, but to me it is an
> > > inevitability. I spent quite a bit of time getting Apache Infra to
> > > enable Buildkite for apache/* repositories precisely so that we have a
> > > coherent path. I hope to be able to have folks on my team focus on
> > > this at some point and contribute it to the community once we have the
> > > bandwidth.
> > >
> > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > > <sz...@gmail.com> wrote:
> > > >
> > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com>
> wrote:
> > > > >
> > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > > <sz...@gmail.com> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm concerned about the overall feedback time we have on pull
> > > requests.
> > > > > > I have a simple PR to make the comment bot working again, but no
> > > > > > builds are running even after 30 minutes.
> > > > >
> > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > > > > same problem sometimes with pull requests taking long to build, but
> > > > > this usually happens when there are large jobs running under my
> name
> > > > > in completely unrelated projects. Also note that if you use e.g.
> > > > > scheduled (cron) actions, then the jobs are executed in the queue
> of
> > > > > the github user who last touched the yml file for that workflow.
> > > > Based on the "Actions" tab the queueing seems chronological to me,
> > > > although I can imagine the behaviour you described.
> > > >
> > > > Has anyone experienced similar clogging on apache/arrow's gha queue
> > > lately?
> > >
>
Re: Github Actions feedback time
Posted by Wes McKinney <we...@gmail.com>.
hi Jorge -- as far as I know, all that's necessary to use BK is the
pipeline configuration in the repo and to attach workers to Arrow's
(free) account (which I'm happy to add you to). We can likely get a
lot of free cloud credits from Google, Microsoft, or Amazon if we
wanted to go that route to keep costs down (I could help pay for the
cloud bill, too).
The one thing to be aware of with BK is the possible security risks
from builds running on pull requests. With reasonable security
practices (not putting sensitive data on the build hosts) this should
be too much of a problem.
On Tue, Jan 5, 2021 at 11:46 AM Jorge Cardoso Leitão
<jo...@gmail.com> wrote:
>
> Krisztián,
>
> I agree with you that there is an ongoing problem with the queue. Thanks
> for the tips wrt to WIP.
>
> Wes, I would be up for moving Rust workflows to Buildkite. Is the
> integration in place wrt to reporting to triggers and reporting back to
> gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there
> any PMC-specific activity that is blocking us from working on this?
>
> Best,
> Jorge
>
>
>
>
> On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com> wrote:
>
> > At the risk of sounding like a broken record -- we are almost
> > certainly going to have to move our builds to dedicated infrastructure
> > that this community has complete agency over sometime between now and
> > 2025. Maybe it will be this year, maybe next year, but to me it is an
> > inevitability. I spent quite a bit of time getting Apache Infra to
> > enable Buildkite for apache/* repositories precisely so that we have a
> > coherent path. I hope to be able to have folks on my team focus on
> > this at some point and contribute it to the community once we have the
> > bandwidth.
> >
> > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> > <sz...@gmail.com> wrote:
> > >
> > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com> wrote:
> > > >
> > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > > <sz...@gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'm concerned about the overall feedback time we have on pull
> > requests.
> > > > > I have a simple PR to make the comment bot working again, but no
> > > > > builds are running even after 30 minutes.
> > > >
> > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > > > same problem sometimes with pull requests taking long to build, but
> > > > this usually happens when there are large jobs running under my name
> > > > in completely unrelated projects. Also note that if you use e.g.
> > > > scheduled (cron) actions, then the jobs are executed in the queue of
> > > > the github user who last touched the yml file for that workflow.
> > > Based on the "Actions" tab the queueing seems chronological to me,
> > > although I can imagine the behaviour you described.
> > >
> > > Has anyone experienced similar clogging on apache/arrow's gha queue
> > lately?
> >
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Krisztián,
I agree with you that there is an ongoing problem with the queue. Thanks
for the tips wrt to WIP.
Wes, I would be up for moving Rust workflows to Buildkite. Is the
integration in place wrt to reporting to triggers and reporting back to
gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there
any PMC-specific activity that is blocking us from working on this?
Best,
Jorge
On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <we...@gmail.com> wrote:
> At the risk of sounding like a broken record -- we are almost
> certainly going to have to move our builds to dedicated infrastructure
> that this community has complete agency over sometime between now and
> 2025. Maybe it will be this year, maybe next year, but to me it is an
> inevitability. I spent quite a bit of time getting Apache Infra to
> enable Buildkite for apache/* repositories precisely so that we have a
> coherent path. I hope to be able to have folks on my team focus on
> this at some point and contribute it to the community once we have the
> bandwidth.
>
> On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
> <sz...@gmail.com> wrote:
> >
> > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com> wrote:
> > >
> > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > > <sz...@gmail.com> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I'm concerned about the overall feedback time we have on pull
> requests.
> > > > I have a simple PR to make the comment bot working again, but no
> > > > builds are running even after 30 minutes.
> > >
> > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > > same problem sometimes with pull requests taking long to build, but
> > > this usually happens when there are large jobs running under my name
> > > in completely unrelated projects. Also note that if you use e.g.
> > > scheduled (cron) actions, then the jobs are executed in the queue of
> > > the github user who last touched the yml file for that workflow.
> > Based on the "Actions" tab the queueing seems chronological to me,
> > although I can imagine the behaviour you described.
> >
> > Has anyone experienced similar clogging on apache/arrow's gha queue
> lately?
>
Re: Github Actions feedback time
Posted by Wes McKinney <we...@gmail.com>.
At the risk of sounding like a broken record -- we are almost
certainly going to have to move our builds to dedicated infrastructure
that this community has complete agency over sometime between now and
2025. Maybe it will be this year, maybe next year, but to me it is an
inevitability. I spent quite a bit of time getting Apache Infra to
enable Buildkite for apache/* repositories precisely so that we have a
coherent path. I hope to be able to have folks on my team focus on
this at some point and contribute it to the community once we have the
bandwidth.
On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs
<sz...@gmail.com> wrote:
>
> On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com> wrote:
> >
> > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> > <sz...@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I'm concerned about the overall feedback time we have on pull requests.
> > > I have a simple PR to make the comment bot working again, but no
> > > builds are running even after 30 minutes.
> >
> > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> > same problem sometimes with pull requests taking long to build, but
> > this usually happens when there are large jobs running under my name
> > in completely unrelated projects. Also note that if you use e.g.
> > scheduled (cron) actions, then the jobs are executed in the queue of
> > the github user who last touched the yml file for that workflow.
> Based on the "Actions" tab the queueing seems chronological to me,
> although I can imagine the behaviour you described.
>
> Has anyone experienced similar clogging on apache/arrow's gha queue lately?
Re: Github Actions feedback time
Posted by Krisztián Szűcs <sz...@gmail.com>.
On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <je...@gmail.com> wrote:
>
> On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
> <sz...@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm concerned about the overall feedback time we have on pull requests.
> > I have a simple PR to make the comment bot working again, but no
> > builds are running even after 30 minutes.
>
> FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
> same problem sometimes with pull requests taking long to build, but
> this usually happens when there are large jobs running under my name
> in completely unrelated projects. Also note that if you use e.g.
> scheduled (cron) actions, then the jobs are executed in the queue of
> the github user who last touched the yml file for that workflow.
Based on the "Actions" tab the queueing seems chronological to me,
although I can imagine the behaviour you described.
Has anyone experienced similar clogging on apache/arrow's gha queue lately?
Re: Github Actions feedback time
Posted by Jeroen Ooms <je...@gmail.com>.
On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs
<sz...@gmail.com> wrote:
>
> Hi,
>
> I'm concerned about the overall feedback time we have on pull requests.
> I have a simple PR to make the comment bot working again, but no
> builds are running even after 30 minutes.
FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the
same problem sometimes with pull requests taking long to build, but
this usually happens when there are large jobs running under my name
in completely unrelated projects. Also note that if you use e.g.
scheduled (cron) actions, then the jobs are executed in the queue of
the github user who last touched the yml file for that workflow.
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi Wes,
Just a quick update here.
I spawned a small ubuntu VM on Azure with a mounted volume to try out the
dockerization. I used rootless mode
<https://docs.docker.com/engine/security/rootless/> since we would like to
run PRs but we have no control over what people PR to the repo and the
pipeline is read directly from the PR(*).
This works as intended. It is roughly bootstrapped as:
1. partition the mounted volume
2. symlink the external mount to docker's share/docker
2. add non-root user
3. add authorized_key to non-root user
4. ssh into non-root user (why
<https://unix.stackexchange.com/a/593756/36325>)
5. install rootless docker
6. enable docker via systemctl --user
7. start build-agent using a token
I have the exact commands I used on each step, if needed. There are still
some improvements, but IMO it is a starting point.
The build is on-going here
<https://buildkite.com/test-811/example/builds/85#>.
(*) We can revisit this later. This currently fails to run Rust's coverage,
which requires `docker run --privileged`.
Best,
Jorge
On Fri, Jan 8, 2021 at 12:38 AM Wes McKinney <we...@gmail.com> wrote:
> Good point re re-use of our other Docker images. Probably the best
> path would be to attach a few bare-metal machines or spin up some
> persistent cloud instances (if we figure out how they're going to be
> paid for).
>
> On Thu, Jan 7, 2021 at 2:15 PM Jorge Cardoso Leitão
> <jo...@gmail.com> wrote:
> >
> > Hi Wes,
> >
> > Thanks for the suggestion and for the offer; much appreciated. I was able
> > to run it on my own macbook on the dockerized agent
> > <https://buildkite.com/docs/agent/v3/docker#running-via-docker>, and it
> > just ran out of the box.
> >
> > The main limitation of this is that it is more difficult to run our own
> > docker images. We can probably make it work via docker on docker,
> > but it is likely more evolved (I haven't tried that yet). We could also
> > build an image on top of "buildkite/agent:3" for this, but well...
> >
> > Best,
> > Jorge
> >
> >
> > On Thu, Jan 7, 2021 at 8:34 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > > Jorge -- if you want to test a Linux agent, you could run the
> > > buildkite-agent in a Docker container. We (Ursa) could possibly look
> > > into adding Dockerized agents on some of our physical machines,
> > > particularly if we set up a well-documented procedure for setting this
> > > up on a new machine.
> > >
> > > On Wed, Jan 6, 2021 at 11:29 PM Jorge Cardoso Leitão
> > > <jo...@gmail.com> wrote:
> > > >
> > > > Hi Jacob,
> > > >
> > > > Neal already requested those (and other) actions to be whitelisted
> here
> > > > https://issues.apache.org/jira/browse/INFRA-21239, but there was no
> > > > response yet.
> > > >
> > > > Best,
> > > > Jorge
> > > >
> > > >
> > > > On Thu, Jan 7, 2021 at 6:20 AM Jacob Quinn <qu...@gmail.com>
> > > wrote:
> > > >
> > > > > From this page, it looks like there have been certain github
> > > organizations
> > > > > that have been "whitelisted" to allow their github actions to run.
> Is
> > > there
> > > > > a process to do this whitelisting? If the `julia-actions` github
> org
> > > was
> > > > > allowed to run, that would enable everything needed for Julia CI to
> > > run.
> > > > >
> > > > > -Jacob
> > > > >
> > > > > On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com>
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > > I wasn't following the build queue's state lately, but I think
> we
> > > > > > > should consolidate the build configurations.
> > > > > > > Possible candidates are the PR* workflows
> > > > > >
> > > > > > https://github.com/apache/arrow/pull/9120
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > --
> > > > > > kou
> > > > > >
> > > > > > In <
> > > CAHM19a7v+mcLVzMU1JtuzTojSBTgdAzCM5NSWReFjW2W0+C14g@mail.gmail.com>
> > > > > > "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38
> +0100,
> > > > > > Krisztián Szűcs <sz...@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I'm concerned about the overall feedback time we have on pull
> > > requests.
> > > > > > > I have a simple PR to make the comment bot working again, but
> no
> > > > > > > builds are running even after 30 minutes.
> > > > > > > I can see 2-4 running builds, which will make our work much
> harder
> > > > > > > right before the release.
> > > > > > >
> > > > > > > I wasn't following the build queue's state lately, but I think
> we
> > > > > > > should consolidate the build configurations.
> > > > > > > Possible candidates are the PR* workflows and good to have
> tests
> > > which
> > > > > > > we could trigger on master instead.
> > > > > > >
> > > > > > > Opinions?
> > > > > > >
> > > > > > > Regards, Krisztian
> > > > > >
> > > > >
> > >
>
Re: Github Actions feedback time
Posted by Wes McKinney <we...@gmail.com>.
Good point re re-use of our other Docker images. Probably the best
path would be to attach a few bare-metal machines or spin up some
persistent cloud instances (if we figure out how they're going to be
paid for).
On Thu, Jan 7, 2021 at 2:15 PM Jorge Cardoso Leitão
<jo...@gmail.com> wrote:
>
> Hi Wes,
>
> Thanks for the suggestion and for the offer; much appreciated. I was able
> to run it on my own macbook on the dockerized agent
> <https://buildkite.com/docs/agent/v3/docker#running-via-docker>, and it
> just ran out of the box.
>
> The main limitation of this is that it is more difficult to run our own
> docker images. We can probably make it work via docker on docker,
> but it is likely more evolved (I haven't tried that yet). We could also
> build an image on top of "buildkite/agent:3" for this, but well...
>
> Best,
> Jorge
>
>
> On Thu, Jan 7, 2021 at 8:34 PM Wes McKinney <we...@gmail.com> wrote:
>
> > Jorge -- if you want to test a Linux agent, you could run the
> > buildkite-agent in a Docker container. We (Ursa) could possibly look
> > into adding Dockerized agents on some of our physical machines,
> > particularly if we set up a well-documented procedure for setting this
> > up on a new machine.
> >
> > On Wed, Jan 6, 2021 at 11:29 PM Jorge Cardoso Leitão
> > <jo...@gmail.com> wrote:
> > >
> > > Hi Jacob,
> > >
> > > Neal already requested those (and other) actions to be whitelisted here
> > > https://issues.apache.org/jira/browse/INFRA-21239, but there was no
> > > response yet.
> > >
> > > Best,
> > > Jorge
> > >
> > >
> > > On Thu, Jan 7, 2021 at 6:20 AM Jacob Quinn <qu...@gmail.com>
> > wrote:
> > >
> > > > From this page, it looks like there have been certain github
> > organizations
> > > > that have been "whitelisted" to allow their github actions to run. Is
> > there
> > > > a process to do this whitelisting? If the `julia-actions` github org
> > was
> > > > allowed to run, that would enable everything needed for Julia CI to
> > run.
> > > >
> > > > -Jacob
> > > >
> > > > On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com>
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > > I wasn't following the build queue's state lately, but I think we
> > > > > > should consolidate the build configurations.
> > > > > > Possible candidates are the PR* workflows
> > > > >
> > > > > https://github.com/apache/arrow/pull/9120
> > > > >
> > > > >
> > > > > Thanks,
> > > > > --
> > > > > kou
> > > > >
> > > > > In <
> > CAHM19a7v+mcLVzMU1JtuzTojSBTgdAzCM5NSWReFjW2W0+C14g@mail.gmail.com>
> > > > > "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
> > > > > Krisztián Szűcs <sz...@gmail.com> wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I'm concerned about the overall feedback time we have on pull
> > requests.
> > > > > > I have a simple PR to make the comment bot working again, but no
> > > > > > builds are running even after 30 minutes.
> > > > > > I can see 2-4 running builds, which will make our work much harder
> > > > > > right before the release.
> > > > > >
> > > > > > I wasn't following the build queue's state lately, but I think we
> > > > > > should consolidate the build configurations.
> > > > > > Possible candidates are the PR* workflows and good to have tests
> > which
> > > > > > we could trigger on master instead.
> > > > > >
> > > > > > Opinions?
> > > > > >
> > > > > > Regards, Krisztian
> > > > >
> > > >
> >
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi Wes,
Thanks for the suggestion and for the offer; much appreciated. I was able
to run it on my own macbook on the dockerized agent
<https://buildkite.com/docs/agent/v3/docker#running-via-docker>, and it
just ran out of the box.
The main limitation of this is that it is more difficult to run our own
docker images. We can probably make it work via docker on docker,
but it is likely more evolved (I haven't tried that yet). We could also
build an image on top of "buildkite/agent:3" for this, but well...
Best,
Jorge
On Thu, Jan 7, 2021 at 8:34 PM Wes McKinney <we...@gmail.com> wrote:
> Jorge -- if you want to test a Linux agent, you could run the
> buildkite-agent in a Docker container. We (Ursa) could possibly look
> into adding Dockerized agents on some of our physical machines,
> particularly if we set up a well-documented procedure for setting this
> up on a new machine.
>
> On Wed, Jan 6, 2021 at 11:29 PM Jorge Cardoso Leitão
> <jo...@gmail.com> wrote:
> >
> > Hi Jacob,
> >
> > Neal already requested those (and other) actions to be whitelisted here
> > https://issues.apache.org/jira/browse/INFRA-21239, but there was no
> > response yet.
> >
> > Best,
> > Jorge
> >
> >
> > On Thu, Jan 7, 2021 at 6:20 AM Jacob Quinn <qu...@gmail.com>
> wrote:
> >
> > > From this page, it looks like there have been certain github
> organizations
> > > that have been "whitelisted" to allow their github actions to run. Is
> there
> > > a process to do this whitelisting? If the `julia-actions` github org
> was
> > > allowed to run, that would enable everything needed for Julia CI to
> run.
> > >
> > > -Jacob
> > >
> > > On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > > I wasn't following the build queue's state lately, but I think we
> > > > > should consolidate the build configurations.
> > > > > Possible candidates are the PR* workflows
> > > >
> > > > https://github.com/apache/arrow/pull/9120
> > > >
> > > >
> > > > Thanks,
> > > > --
> > > > kou
> > > >
> > > > In <
> CAHM19a7v+mcLVzMU1JtuzTojSBTgdAzCM5NSWReFjW2W0+C14g@mail.gmail.com>
> > > > "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
> > > > Krisztián Szűcs <sz...@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I'm concerned about the overall feedback time we have on pull
> requests.
> > > > > I have a simple PR to make the comment bot working again, but no
> > > > > builds are running even after 30 minutes.
> > > > > I can see 2-4 running builds, which will make our work much harder
> > > > > right before the release.
> > > > >
> > > > > I wasn't following the build queue's state lately, but I think we
> > > > > should consolidate the build configurations.
> > > > > Possible candidates are the PR* workflows and good to have tests
> which
> > > > > we could trigger on master instead.
> > > > >
> > > > > Opinions?
> > > > >
> > > > > Regards, Krisztian
> > > >
> > >
>
Re: Github Actions feedback time
Posted by Wes McKinney <we...@gmail.com>.
Jorge -- if you want to test a Linux agent, you could run the
buildkite-agent in a Docker container. We (Ursa) could possibly look
into adding Dockerized agents on some of our physical machines,
particularly if we set up a well-documented procedure for setting this
up on a new machine.
On Wed, Jan 6, 2021 at 11:29 PM Jorge Cardoso Leitão
<jo...@gmail.com> wrote:
>
> Hi Jacob,
>
> Neal already requested those (and other) actions to be whitelisted here
> https://issues.apache.org/jira/browse/INFRA-21239, but there was no
> response yet.
>
> Best,
> Jorge
>
>
> On Thu, Jan 7, 2021 at 6:20 AM Jacob Quinn <qu...@gmail.com> wrote:
>
> > From this page, it looks like there have been certain github organizations
> > that have been "whitelisted" to allow their github actions to run. Is there
> > a process to do this whitelisting? If the `julia-actions` github org was
> > allowed to run, that would enable everything needed for Julia CI to run.
> >
> > -Jacob
> >
> > On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> >
> > > Hi,
> > >
> > > > I wasn't following the build queue's state lately, but I think we
> > > > should consolidate the build configurations.
> > > > Possible candidates are the PR* workflows
> > >
> > > https://github.com/apache/arrow/pull/9120
> > >
> > >
> > > Thanks,
> > > --
> > > kou
> > >
> > > In <CA...@mail.gmail.com>
> > > "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
> > > Krisztián Szűcs <sz...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm concerned about the overall feedback time we have on pull requests.
> > > > I have a simple PR to make the comment bot working again, but no
> > > > builds are running even after 30 minutes.
> > > > I can see 2-4 running builds, which will make our work much harder
> > > > right before the release.
> > > >
> > > > I wasn't following the build queue's state lately, but I think we
> > > > should consolidate the build configurations.
> > > > Possible candidates are the PR* workflows and good to have tests which
> > > > we could trigger on master instead.
> > > >
> > > > Opinions?
> > > >
> > > > Regards, Krisztian
> > >
> >
Re: Github Actions feedback time
Posted by Jorge Cardoso Leitão <jo...@gmail.com>.
Hi Jacob,
Neal already requested those (and other) actions to be whitelisted here
https://issues.apache.org/jira/browse/INFRA-21239, but there was no
response yet.
Best,
Jorge
On Thu, Jan 7, 2021 at 6:20 AM Jacob Quinn <qu...@gmail.com> wrote:
> From this page, it looks like there have been certain github organizations
> that have been "whitelisted" to allow their github actions to run. Is there
> a process to do this whitelisting? If the `julia-actions` github org was
> allowed to run, that would enable everything needed for Julia CI to run.
>
> -Jacob
>
> On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> > Hi,
> >
> > > I wasn't following the build queue's state lately, but I think we
> > > should consolidate the build configurations.
> > > Possible candidates are the PR* workflows
> >
> > https://github.com/apache/arrow/pull/9120
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In <CA...@mail.gmail.com>
> > "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
> > Krisztián Szűcs <sz...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I'm concerned about the overall feedback time we have on pull requests.
> > > I have a simple PR to make the comment bot working again, but no
> > > builds are running even after 30 minutes.
> > > I can see 2-4 running builds, which will make our work much harder
> > > right before the release.
> > >
> > > I wasn't following the build queue's state lately, but I think we
> > > should consolidate the build configurations.
> > > Possible candidates are the PR* workflows and good to have tests which
> > > we could trigger on master instead.
> > >
> > > Opinions?
> > >
> > > Regards, Krisztian
> >
>
Re: Github Actions feedback time
Posted by Jacob Quinn <qu...@gmail.com>.
From this page, it looks like there have been certain github organizations
that have been "whitelisted" to allow their github actions to run. Is there
a process to do this whitelisting? If the `julia-actions` github org was
allowed to run, that would enable everything needed for Julia CI to run.
-Jacob
On Wed, Jan 6, 2021 at 10:00 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
> > I wasn't following the build queue's state lately, but I think we
> > should consolidate the build configurations.
> > Possible candidates are the PR* workflows
>
> https://github.com/apache/arrow/pull/9120
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
> Krisztián Szűcs <sz...@gmail.com> wrote:
>
> > Hi,
> >
> > I'm concerned about the overall feedback time we have on pull requests.
> > I have a simple PR to make the comment bot working again, but no
> > builds are running even after 30 minutes.
> > I can see 2-4 running builds, which will make our work much harder
> > right before the release.
> >
> > I wasn't following the build queue's state lately, but I think we
> > should consolidate the build configurations.
> > Possible candidates are the PR* workflows and good to have tests which
> > we could trigger on master instead.
> >
> > Opinions?
> >
> > Regards, Krisztian
>
Re: Github Actions feedback time
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> I wasn't following the build queue's state lately, but I think we
> should consolidate the build configurations.
> Possible candidates are the PR* workflows
https://github.com/apache/arrow/pull/9120
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Github Actions feedback time" on Tue, 5 Jan 2021 13:33:38 +0100,
Krisztián Szűcs <sz...@gmail.com> wrote:
> Hi,
>
> I'm concerned about the overall feedback time we have on pull requests.
> I have a simple PR to make the comment bot working again, but no
> builds are running even after 30 minutes.
> I can see 2-4 running builds, which will make our work much harder
> right before the release.
>
> I wasn't following the build queue's state lately, but I think we
> should consolidate the build configurations.
> Possible candidates are the PR* workflows and good to have tests which
> we could trigger on master instead.
>
> Opinions?
>
> Regards, Krisztian