You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Sutou Kouhei <ko...@clear-code.com> on 2021/01/09 23:36:17 UTC
Re: ursa-labs/crossbow on travis-ci.com is disabled
Hi,
Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
($249/month) only for this month? It's just for the 3.0.0
release. We can stop the plan after we release 3.0.0.
See also: https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
Background:
We have only two Crossbow task types that depend on Travis
CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
We'll be able to migrate testing Homebrew on macOS tasks to
other CI like GitHub Actions easily. But building .deb/.rpm
for ARM64 tasks are difficult.
Here are failed candidates:
1. Using GitHub Actions with x86_64 host + QEMU + ccache
We know that building .deb/.rpm for ARM64 with x86_64
host + QEMU takes 6h+.
Hypothesis: If we use ccache and share cache with
builds, we may be able to reduce build time.
Fact: We can't share cache between Crossbow
tasks. Because Crossbow creates a new branch for each
task and doesn't use the default branch. The GitHub
Actions cache feature doesn't share cache between
branches:
https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
2. Using Drone Cloud like our conda-*-aarch64 tasks
Building .deb/.rpm tasks prepare build in base host and
run a Docker container to build .deb/.rpm. We need to
run a Docker container in CI. Travis CI and GitHub
Actions can do it.
It seems that Drone Cloud uses Docker pipeline:
https://docs.drone.io/yaml/docker/
We need Docker-in-Docker support for building .deb/.rpm
tasks. But Drone Cloud doesn't support Docker-in-Docker:
https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
We may be able to use Bulidkite based CI for 4.0.0.
Thanks,
--
kou
In <63...@www.fastmail.com>
"Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov 2020 09:58:59 +0100,
"Uwe L. Korn" <uw...@xhochy.com> wrote:
> Also note that drone.io supports linux-arm64 which we use in conda-forge for this architecture and is already setup in crossbow (although we had issues with branches not being seen).
>
> On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
>> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>> >
>> > Hi,
>> >
>> > In <CA...@mail.gmail.com>
>> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24 Nov 2020 13:36:54 +0100,
>> > Krisztián Szűcs <sz...@gmail.com> wrote:
>> >
>> > > Confirmed, we already have a negative credit balance due to travis'
>> > > new billing strategy.
>> > > The macos wheels quickly consume the credit based free tier, so travis
>> > > disables even the linux builds.
>> > >
>> > > I think we should migrate away from travis to gha or azure, drawbacks:
>> > > - the wheel scripts are tailored for travis
>> > > - only amd64 arch
>> >
>> > Thanks for confirming it.
>> >
>> > It seems that we have 10,000 credits per month. Our
>> > travis-ci.com jobs will be enabled again after 4 days. If we
>> > reduce our travis-ci.com usage, we may be able to keep using
>> > travis.ci.com.
>>
>> I found that if you contact their support, they may be willing to
>> donate much more free credits for selected open source projects. They
>> already mention Apache (the httpd) as an example in their blog of
>> important OSS: https://blog.travis-ci.com/oss-announcement
>>
Re: ursa-labs/crossbow on travis-ci.com is disabled
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
We released 3.0.0. We can stop the Travis CI subscription. Thanks!
We should prepare new (Buildkite based?) CI for arm64 by the
next release.
Thanks,
--
kou
In <20...@clear-code.com>
"Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 12 Jan 2021 04:00:15 +0900 (JST),
Sutou Kouhei <ko...@clear-code.com> wrote:
> Thanks!!!
>
> In <CA...@mail.gmail.com>
> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Mon, 11 Jan 2021 07:58:53 -0600,
> Wes McKinney <we...@gmail.com> wrote:
>
>> hi Kou — I just activated a 5 concurrent build Travis CI subscription
>> on @ursacomputing on GitHub (since we are moving some repos there from
>> ursa-labs, and that is also a paid GitHub organization). We can create
>> a team for Arrow collaborators to give push access to repos such as
>> crossbow
>>
>> I understand that ursa-labs/crossbow was recently disabled by GitHub
>> due to the large number of releas artifacts, so a new repo may need to
>> be created there. I will leave it to Krisztian or others to sort out.
>>
>> Thanks,
>> Wes
>>
>> On Sat, Jan 9, 2021 at 5:44 PM Wes McKinney <we...@gmail.com> wrote:
>>>
>>> Hi Kou — yes, we can do that. I’m not able to do it right this minute but I can look in a few hours or Neal may beat me to it.
>>>
>>> On Sat, Jan 9, 2021 at 5:36 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
>>>> ($249/month) only for this month? It's just for the 3.0.0
>>>> release. We can stop the plan after we release 3.0.0.
>>>>
>>>> See also: https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
>>>>
>>>> Background:
>>>>
>>>> We have only two Crossbow task types that depend on Travis
>>>> CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
>>>>
>>>> We'll be able to migrate testing Homebrew on macOS tasks to
>>>> other CI like GitHub Actions easily. But building .deb/.rpm
>>>> for ARM64 tasks are difficult.
>>>>
>>>> Here are failed candidates:
>>>>
>>>> 1. Using GitHub Actions with x86_64 host + QEMU + ccache
>>>>
>>>> We know that building .deb/.rpm for ARM64 with x86_64
>>>> host + QEMU takes 6h+.
>>>>
>>>> Hypothesis: If we use ccache and share cache with
>>>> builds, we may be able to reduce build time.
>>>>
>>>> Fact: We can't share cache between Crossbow
>>>> tasks. Because Crossbow creates a new branch for each
>>>> task and doesn't use the default branch. The GitHub
>>>> Actions cache feature doesn't share cache between
>>>> branches:
>>>> https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
>>>>
>>>> 2. Using Drone Cloud like our conda-*-aarch64 tasks
>>>>
>>>> Building .deb/.rpm tasks prepare build in base host and
>>>> run a Docker container to build .deb/.rpm. We need to
>>>> run a Docker container in CI. Travis CI and GitHub
>>>> Actions can do it.
>>>>
>>>> It seems that Drone Cloud uses Docker pipeline:
>>>> https://docs.drone.io/yaml/docker/
>>>> We need Docker-in-Docker support for building .deb/.rpm
>>>> tasks. But Drone Cloud doesn't support Docker-in-Docker:
>>>> https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
>>>>
>>>>
>>>> We may be able to use Bulidkite based CI for 4.0.0.
>>>>
>>>>
>>>> Thanks,
>>>> --
>>>> kou
>>>>
>>>> In <63...@www.fastmail.com>
>>>> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov 2020 09:58:59 +0100,
>>>> "Uwe L. Korn" <uw...@xhochy.com> wrote:
>>>>
>>>> > Also note that drone.io supports linux-arm64 which we use in conda-forge for this architecture and is already setup in crossbow (although we had issues with branches not being seen).
>>>> >
>>>> > On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
>>>> >> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>>> >> >
>>>> >> > Hi,
>>>> >> >
>>>> >> > In <CA...@mail.gmail.com>
>>>> >> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24 Nov 2020 13:36:54 +0100,
>>>> >> > Krisztián Szűcs <sz...@gmail.com> wrote:
>>>> >> >
>>>> >> > > Confirmed, we already have a negative credit balance due to travis'
>>>> >> > > new billing strategy.
>>>> >> > > The macos wheels quickly consume the credit based free tier, so travis
>>>> >> > > disables even the linux builds.
>>>> >> > >
>>>> >> > > I think we should migrate away from travis to gha or azure, drawbacks:
>>>> >> > > - the wheel scripts are tailored for travis
>>>> >> > > - only amd64 arch
>>>> >> >
>>>> >> > Thanks for confirming it.
>>>> >> >
>>>> >> > It seems that we have 10,000 credits per month. Our
>>>> >> > travis-ci.com jobs will be enabled again after 4 days. If we
>>>> >> > reduce our travis-ci.com usage, we may be able to keep using
>>>> >> > travis.ci.com.
>>>> >>
>>>> >> I found that if you contact their support, they may be willing to
>>>> >> donate much more free credits for selected open source projects. They
>>>> >> already mention Apache (the httpd) as an example in their blog of
>>>> >> important OSS: https://blog.travis-ci.com/oss-announcement
>>>> >>
Re: ursa-labs/crossbow on travis-ci.com is disabled
Posted by Krisztián Szűcs <sz...@gmail.com>.
I migrated the CI configurations from ursa-labs/crossbow to
ursacomputing/crossbow [1] which should be the default after merging
[2].
I will still need to configure additional credentials for publishing
nightly artifacts, but the next crossbow runs should pass.
In the meantime you should be able to trigger builds using the comment
bot (but gha is really slow):
@github-actions crossbow -c ursacomputing/crossbow submit -g wheel
Or locally using the crossbow.py script directly.
Here is an ARM build:
https://travis-ci.com/github/ursacomputing/crossbow/builds/212762113
[1]: https://github.com/ursacomputing/crossbow
[2]: https://github.com/apache/arrow/pull/9096
On Mon, Jan 11, 2021 at 8:00 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> Thanks!!!
>
> In <CA...@mail.gmail.com>
> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Mon, 11 Jan 2021 07:58:53 -0600,
> Wes McKinney <we...@gmail.com> wrote:
>
> > hi Kou — I just activated a 5 concurrent build Travis CI subscription
> > on @ursacomputing on GitHub (since we are moving some repos there from
> > ursa-labs, and that is also a paid GitHub organization). We can create
> > a team for Arrow collaborators to give push access to repos such as
> > crossbow
> >
> > I understand that ursa-labs/crossbow was recently disabled by GitHub
> > due to the large number of releas artifacts, so a new repo may need to
> > be created there. I will leave it to Krisztian or others to sort out.
> >
> > Thanks,
> > Wes
> >
> > On Sat, Jan 9, 2021 at 5:44 PM Wes McKinney <we...@gmail.com> wrote:
> >>
> >> Hi Kou — yes, we can do that. I’m not able to do it right this minute but I can look in a few hours or Neal may beat me to it.
> >>
> >> On Sat, Jan 9, 2021 at 5:36 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
> >>> ($249/month) only for this month? It's just for the 3.0.0
> >>> release. We can stop the plan after we release 3.0.0.
> >>>
> >>> See also: https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
> >>>
> >>> Background:
> >>>
> >>> We have only two Crossbow task types that depend on Travis
> >>> CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
> >>>
> >>> We'll be able to migrate testing Homebrew on macOS tasks to
> >>> other CI like GitHub Actions easily. But building .deb/.rpm
> >>> for ARM64 tasks are difficult.
> >>>
> >>> Here are failed candidates:
> >>>
> >>> 1. Using GitHub Actions with x86_64 host + QEMU + ccache
> >>>
> >>> We know that building .deb/.rpm for ARM64 with x86_64
> >>> host + QEMU takes 6h+.
> >>>
> >>> Hypothesis: If we use ccache and share cache with
> >>> builds, we may be able to reduce build time.
> >>>
> >>> Fact: We can't share cache between Crossbow
> >>> tasks. Because Crossbow creates a new branch for each
> >>> task and doesn't use the default branch. The GitHub
> >>> Actions cache feature doesn't share cache between
> >>> branches:
> >>> https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
> >>>
> >>> 2. Using Drone Cloud like our conda-*-aarch64 tasks
> >>>
> >>> Building .deb/.rpm tasks prepare build in base host and
> >>> run a Docker container to build .deb/.rpm. We need to
> >>> run a Docker container in CI. Travis CI and GitHub
> >>> Actions can do it.
> >>>
> >>> It seems that Drone Cloud uses Docker pipeline:
> >>> https://docs.drone.io/yaml/docker/
> >>> We need Docker-in-Docker support for building .deb/.rpm
> >>> tasks. But Drone Cloud doesn't support Docker-in-Docker:
> >>> https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
> >>>
> >>>
> >>> We may be able to use Bulidkite based CI for 4.0.0.
> >>>
> >>>
> >>> Thanks,
> >>> --
> >>> kou
> >>>
> >>> In <63...@www.fastmail.com>
> >>> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov 2020 09:58:59 +0100,
> >>> "Uwe L. Korn" <uw...@xhochy.com> wrote:
> >>>
> >>> > Also note that drone.io supports linux-arm64 which we use in conda-forge for this architecture and is already setup in crossbow (although we had issues with branches not being seen).
> >>> >
> >>> > On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
> >>> >> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> >>> >> >
> >>> >> > Hi,
> >>> >> >
> >>> >> > In <CA...@mail.gmail.com>
> >>> >> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24 Nov 2020 13:36:54 +0100,
> >>> >> > Krisztián Szűcs <sz...@gmail.com> wrote:
> >>> >> >
> >>> >> > > Confirmed, we already have a negative credit balance due to travis'
> >>> >> > > new billing strategy.
> >>> >> > > The macos wheels quickly consume the credit based free tier, so travis
> >>> >> > > disables even the linux builds.
> >>> >> > >
> >>> >> > > I think we should migrate away from travis to gha or azure, drawbacks:
> >>> >> > > - the wheel scripts are tailored for travis
> >>> >> > > - only amd64 arch
> >>> >> >
> >>> >> > Thanks for confirming it.
> >>> >> >
> >>> >> > It seems that we have 10,000 credits per month. Our
> >>> >> > travis-ci.com jobs will be enabled again after 4 days. If we
> >>> >> > reduce our travis-ci.com usage, we may be able to keep using
> >>> >> > travis.ci.com.
> >>> >>
> >>> >> I found that if you contact their support, they may be willing to
> >>> >> donate much more free credits for selected open source projects. They
> >>> >> already mention Apache (the httpd) as an example in their blog of
> >>> >> important OSS: https://blog.travis-ci.com/oss-announcement
> >>> >>
Re: ursa-labs/crossbow on travis-ci.com is disabled
Posted by Sutou Kouhei <ko...@clear-code.com>.
Thanks!!!
In <CA...@mail.gmail.com>
"Re: ursa-labs/crossbow on travis-ci.com is disabled" on Mon, 11 Jan 2021 07:58:53 -0600,
Wes McKinney <we...@gmail.com> wrote:
> hi Kou — I just activated a 5 concurrent build Travis CI subscription
> on @ursacomputing on GitHub (since we are moving some repos there from
> ursa-labs, and that is also a paid GitHub organization). We can create
> a team for Arrow collaborators to give push access to repos such as
> crossbow
>
> I understand that ursa-labs/crossbow was recently disabled by GitHub
> due to the large number of releas artifacts, so a new repo may need to
> be created there. I will leave it to Krisztian or others to sort out.
>
> Thanks,
> Wes
>
> On Sat, Jan 9, 2021 at 5:44 PM Wes McKinney <we...@gmail.com> wrote:
>>
>> Hi Kou — yes, we can do that. I’m not able to do it right this minute but I can look in a few hours or Neal may beat me to it.
>>
>> On Sat, Jan 9, 2021 at 5:36 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>>
>>> Hi,
>>>
>>> Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
>>> ($249/month) only for this month? It's just for the 3.0.0
>>> release. We can stop the plan after we release 3.0.0.
>>>
>>> See also: https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
>>>
>>> Background:
>>>
>>> We have only two Crossbow task types that depend on Travis
>>> CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
>>>
>>> We'll be able to migrate testing Homebrew on macOS tasks to
>>> other CI like GitHub Actions easily. But building .deb/.rpm
>>> for ARM64 tasks are difficult.
>>>
>>> Here are failed candidates:
>>>
>>> 1. Using GitHub Actions with x86_64 host + QEMU + ccache
>>>
>>> We know that building .deb/.rpm for ARM64 with x86_64
>>> host + QEMU takes 6h+.
>>>
>>> Hypothesis: If we use ccache and share cache with
>>> builds, we may be able to reduce build time.
>>>
>>> Fact: We can't share cache between Crossbow
>>> tasks. Because Crossbow creates a new branch for each
>>> task and doesn't use the default branch. The GitHub
>>> Actions cache feature doesn't share cache between
>>> branches:
>>> https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
>>>
>>> 2. Using Drone Cloud like our conda-*-aarch64 tasks
>>>
>>> Building .deb/.rpm tasks prepare build in base host and
>>> run a Docker container to build .deb/.rpm. We need to
>>> run a Docker container in CI. Travis CI and GitHub
>>> Actions can do it.
>>>
>>> It seems that Drone Cloud uses Docker pipeline:
>>> https://docs.drone.io/yaml/docker/
>>> We need Docker-in-Docker support for building .deb/.rpm
>>> tasks. But Drone Cloud doesn't support Docker-in-Docker:
>>> https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
>>>
>>>
>>> We may be able to use Bulidkite based CI for 4.0.0.
>>>
>>>
>>> Thanks,
>>> --
>>> kou
>>>
>>> In <63...@www.fastmail.com>
>>> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov 2020 09:58:59 +0100,
>>> "Uwe L. Korn" <uw...@xhochy.com> wrote:
>>>
>>> > Also note that drone.io supports linux-arm64 which we use in conda-forge for this architecture and is already setup in crossbow (although we had issues with branches not being seen).
>>> >
>>> > On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
>>> >> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>> >> >
>>> >> > Hi,
>>> >> >
>>> >> > In <CA...@mail.gmail.com>
>>> >> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24 Nov 2020 13:36:54 +0100,
>>> >> > Krisztián Szűcs <sz...@gmail.com> wrote:
>>> >> >
>>> >> > > Confirmed, we already have a negative credit balance due to travis'
>>> >> > > new billing strategy.
>>> >> > > The macos wheels quickly consume the credit based free tier, so travis
>>> >> > > disables even the linux builds.
>>> >> > >
>>> >> > > I think we should migrate away from travis to gha or azure, drawbacks:
>>> >> > > - the wheel scripts are tailored for travis
>>> >> > > - only amd64 arch
>>> >> >
>>> >> > Thanks for confirming it.
>>> >> >
>>> >> > It seems that we have 10,000 credits per month. Our
>>> >> > travis-ci.com jobs will be enabled again after 4 days. If we
>>> >> > reduce our travis-ci.com usage, we may be able to keep using
>>> >> > travis.ci.com.
>>> >>
>>> >> I found that if you contact their support, they may be willing to
>>> >> donate much more free credits for selected open source projects. They
>>> >> already mention Apache (the httpd) as an example in their blog of
>>> >> important OSS: https://blog.travis-ci.com/oss-announcement
>>> >>
Re: ursa-labs/crossbow on travis-ci.com is disabled
Posted by Wes McKinney <we...@gmail.com>.
hi Kou — I just activated a 5 concurrent build Travis CI subscription
on @ursacomputing on GitHub (since we are moving some repos there from
ursa-labs, and that is also a paid GitHub organization). We can create
a team for Arrow collaborators to give push access to repos such as
crossbow
I understand that ursa-labs/crossbow was recently disabled by GitHub
due to the large number of releas artifacts, so a new repo may need to
be created there. I will leave it to Krisztian or others to sort out.
Thanks,
Wes
On Sat, Jan 9, 2021 at 5:44 PM Wes McKinney <we...@gmail.com> wrote:
>
> Hi Kou — yes, we can do that. I’m not able to do it right this minute but I can look in a few hours or Neal may beat me to it.
>
> On Sat, Jan 9, 2021 at 5:36 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>>
>> Hi,
>>
>> Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
>> ($249/month) only for this month? It's just for the 3.0.0
>> release. We can stop the plan after we release 3.0.0.
>>
>> See also: https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
>>
>> Background:
>>
>> We have only two Crossbow task types that depend on Travis
>> CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
>>
>> We'll be able to migrate testing Homebrew on macOS tasks to
>> other CI like GitHub Actions easily. But building .deb/.rpm
>> for ARM64 tasks are difficult.
>>
>> Here are failed candidates:
>>
>> 1. Using GitHub Actions with x86_64 host + QEMU + ccache
>>
>> We know that building .deb/.rpm for ARM64 with x86_64
>> host + QEMU takes 6h+.
>>
>> Hypothesis: If we use ccache and share cache with
>> builds, we may be able to reduce build time.
>>
>> Fact: We can't share cache between Crossbow
>> tasks. Because Crossbow creates a new branch for each
>> task and doesn't use the default branch. The GitHub
>> Actions cache feature doesn't share cache between
>> branches:
>> https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
>>
>> 2. Using Drone Cloud like our conda-*-aarch64 tasks
>>
>> Building .deb/.rpm tasks prepare build in base host and
>> run a Docker container to build .deb/.rpm. We need to
>> run a Docker container in CI. Travis CI and GitHub
>> Actions can do it.
>>
>> It seems that Drone Cloud uses Docker pipeline:
>> https://docs.drone.io/yaml/docker/
>> We need Docker-in-Docker support for building .deb/.rpm
>> tasks. But Drone Cloud doesn't support Docker-in-Docker:
>> https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
>>
>>
>> We may be able to use Bulidkite based CI for 4.0.0.
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <63...@www.fastmail.com>
>> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov 2020 09:58:59 +0100,
>> "Uwe L. Korn" <uw...@xhochy.com> wrote:
>>
>> > Also note that drone.io supports linux-arm64 which we use in conda-forge for this architecture and is already setup in crossbow (although we had issues with branches not being seen).
>> >
>> > On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
>> >> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com> wrote:
>> >> >
>> >> > Hi,
>> >> >
>> >> > In <CA...@mail.gmail.com>
>> >> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24 Nov 2020 13:36:54 +0100,
>> >> > Krisztián Szűcs <sz...@gmail.com> wrote:
>> >> >
>> >> > > Confirmed, we already have a negative credit balance due to travis'
>> >> > > new billing strategy.
>> >> > > The macos wheels quickly consume the credit based free tier, so travis
>> >> > > disables even the linux builds.
>> >> > >
>> >> > > I think we should migrate away from travis to gha or azure, drawbacks:
>> >> > > - the wheel scripts are tailored for travis
>> >> > > - only amd64 arch
>> >> >
>> >> > Thanks for confirming it.
>> >> >
>> >> > It seems that we have 10,000 credits per month. Our
>> >> > travis-ci.com jobs will be enabled again after 4 days. If we
>> >> > reduce our travis-ci.com usage, we may be able to keep using
>> >> > travis.ci.com.
>> >>
>> >> I found that if you contact their support, they may be willing to
>> >> donate much more free credits for selected open source projects. They
>> >> already mention Apache (the httpd) as an example in their blog of
>> >> important OSS: https://blog.travis-ci.com/oss-announcement
>> >>
Re: ursa-labs/crossbow on travis-ci.com is disabled
Posted by Wes McKinney <we...@gmail.com>.
Hi Kou — yes, we can do that. I’m not able to do it right this minute but I
can look in a few hours or Neal may beat me to it.
On Sat, Jan 9, 2021 at 5:36 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
> Could Ursa Labs buy the Travis CI's "5 Concurrent plan"
> ($249/month) only for this month? It's just for the 3.0.0
> release. We can stop the plan after we release 3.0.0.
>
> See also:
> https://docs.travis-ci.com/user/billing-overview/#concurrency-based-plans
>
> Background:
>
> We have only two Crossbow task types that depend on Travis
> CI: building .deb/.rpm for ARM64 and testing Homebrew on macOS
>
> We'll be able to migrate testing Homebrew on macOS tasks to
> other CI like GitHub Actions easily. But building .deb/.rpm
> for ARM64 tasks are difficult.
>
> Here are failed candidates:
>
> 1. Using GitHub Actions with x86_64 host + QEMU + ccache
>
> We know that building .deb/.rpm for ARM64 with x86_64
> host + QEMU takes 6h+.
>
> Hypothesis: If we use ccache and share cache with
> builds, we may be able to reduce build time.
>
> Fact: We can't share cache between Crossbow
> tasks. Because Crossbow creates a new branch for each
> task and doesn't use the default branch. The GitHub
> Actions cache feature doesn't share cache between
> branches:
>
> https://docs.github.com/en/free-pro-team@latest/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key
>
> 2. Using Drone Cloud like our conda-*-aarch64 tasks
>
> Building .deb/.rpm tasks prepare build in base host and
> run a Docker container to build .deb/.rpm. We need to
> run a Docker container in CI. Travis CI and GitHub
> Actions can do it.
>
> It seems that Drone Cloud uses Docker pipeline:
> https://docs.drone.io/yaml/docker/
> We need Docker-in-Docker support for building .deb/.rpm
> tasks. But Drone Cloud doesn't support Docker-in-Docker:
> https://discourse.drone.io/t/run-docker-container-in-docker/5704/2
>
>
> We may be able to use Bulidkite based CI for 4.0.0.
>
>
> Thanks,
> --
> kou
>
> In <63...@www.fastmail.com>
> "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Thu, 26 Nov
> 2020 09:58:59 +0100,
> "Uwe L. Korn" <uw...@xhochy.com> wrote:
>
> > Also note that drone.io supports linux-arm64 which we use in
> conda-forge for this architecture and is already setup in crossbow
> (although we had issues with branches not being seen).
> >
> > On Thu, Nov 26, 2020, at 1:31 AM, Jeroen Ooms wrote:
> >> On Wed, Nov 25, 2020 at 10:54 PM Sutou Kouhei <ko...@clear-code.com>
> wrote:
> >> >
> >> > Hi,
> >> >
> >> > In <CAHM19a6BCyWVmD9=NvvzW8HU4tWgm=
> nJOHy+ivSjOtQofnokmw@mail.gmail.com>
> >> > "Re: ursa-labs/crossbow on travis-ci.com is disabled" on Tue, 24
> Nov 2020 13:36:54 +0100,
> >> > Krisztián Szűcs <sz...@gmail.com> wrote:
> >> >
> >> > > Confirmed, we already have a negative credit balance due to travis'
> >> > > new billing strategy.
> >> > > The macos wheels quickly consume the credit based free tier, so
> travis
> >> > > disables even the linux builds.
> >> > >
> >> > > I think we should migrate away from travis to gha or azure,
> drawbacks:
> >> > > - the wheel scripts are tailored for travis
> >> > > - only amd64 arch
> >> >
> >> > Thanks for confirming it.
> >> >
> >> > It seems that we have 10,000 credits per month. Our
> >> > travis-ci.com jobs will be enabled again after 4 days. If we
> >> > reduce our travis-ci.com usage, we may be able to keep using
> >> > travis.ci.com.
> >>
> >> I found that if you contact their support, they may be willing to
> >> donate much more free credits for selected open source projects. They
> >> already mention Apache (the httpd) as an example in their blog of
> >> important OSS: https://blog.travis-ci.com/oss-announcement
> >>
>