CI

[Aristedes Maniatis <ar...@ish.com.au.INVALID>]

On 10/10/21 5:26pm, Andrus Adamchik wrote:
> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.

I've got a bit of experience [1] with github actions in some of my other 
open source projects, and mostly they are very good. Ties in nicely with 
pull requests as well. However a bit more limited with regard to 
database choices. Ideally we'd want databases like SQL Server, AWS 
Aurora, etc in our test suite.

Are we limited at all within the Apache project with how we can 
configure Actions?

Ari


[1] https://github.com/ishgroup/oncourse/actions

Re: CI

[Aristedes Maniatis <ar...@ish.com.au.INVALID>]

On 15/10/21 12:00am, Andrus Adamchik wrote:
> How good is buildbot? Is it capable of running dockerized builds? Does it have MacOS and Windows agents?

Buildbot is great as a tool. I use it myself outside Apache. But I don't 
think the Apache installation is going to give us containers with SQL 
Server, etc.


> My (uninformed) vote is to abandon buildbot and stay on Travis. When/if it becomes untenable, migrate to GitHub Actions outside Apache GitHub (or maybe within Apache if GA restrictions are removed).
>
> Automating release artifacts is a good idea. But probably not until we are more confident in our CI infrastructure.


I agree with all this. Unless anyone has any other thoughts, I'll let 
infra know to kill our old buildbot instance (which isn't working anyway).


Ari



>
> Andrus
>
>
>
>> On Oct 14, 2021, at 8:20 AM, Aristedes Maniatis <ar...@ish.com.au.INVALID> wrote:
>>
>> Should we migrate the buildbot builder on Apache infrastructure over to git and get it working again? Or should we put more effort into Travis and just abandon the Apache builder?
>>
>> Are the new limits at Travis going to work for us?
>>
>> I think we'll be more able to set up different database containers at Travis rather than at Apache.
>>
>>
>> Are we happy with creating release artifacts:
>>
>> a. as we do today, built on a local developer machine
>> b. from Travis
>> c. from buildbot
>>
>>
>> https://issues.apache.org/jira/browse/INFRA-22396
>>
>>
>> Ari
>>
>>
>> On 10/10/21 6:38pm, Andrus Adamchik wrote:
>>> Oops. I guess we shouldn't jump to GA just yet.
>>>
>>>> On Oct 10, 2021, at 10:28 AM, Nikita Timofeev <nt...@objectstyle.com> wrote:
>>>>
>>>>
>>>> Hi all,
>>>> I've read about limitations on the Apache side. It turns out not so great in that case. There's only approved actions available and, more important, there's limited number of runners available for *all* Apache projects. Ifra warns that it could be really slow [1]
>>>> [1] https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=173085013#content/view/173085013 Sunday, 10 October 2021, 10:10am +03:00 from Andrus Adamchik  aadamchik@gmail.com :
>>>>
>>>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>>>
>>>>>
>>>>> Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.
>>>>>
>>>>> However a bit more limited with regard to database choices.
>>>>> We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).
>>>>>
>>>>> Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.
>>>>>
>>>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>>>
>>>>> Haven't tried it, good question. I hope not and it is all straight GitHub.
>>>>>
>>>>> Andrus
>>>>>
>>>>>
>>>>> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis < ari@ish.com.au.INVALID> wrote:
>>>>>
>>>>> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>>>>>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
>>>>> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
>>>>>
>>>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>>>
>>>>> Ari
>>>>>
>>>>>
>>>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>>>

Re: CI

[Andrus Adamchik <aa...@gmail.com>]

How good is buildbot? Is it capable of running dockerized builds? Does it have MacOS and Windows agents?

My (uninformed) vote is to abandon buildbot and stay on Travis. When/if it becomes untenable, migrate to GitHub Actions outside Apache GitHub (or maybe within Apache if GA restrictions are removed).

Automating release artifacts is a good idea. But probably not until we are more confident in our CI infrastructure.

Andrus



> On Oct 14, 2021, at 8:20 AM, Aristedes Maniatis <ar...@ish.com.au.INVALID> wrote:
> 
> Should we migrate the buildbot builder on Apache infrastructure over to git and get it working again? Or should we put more effort into Travis and just abandon the Apache builder?
> 
> Are the new limits at Travis going to work for us?
> 
> I think we'll be more able to set up different database containers at Travis rather than at Apache.
> 
> 
> Are we happy with creating release artifacts:
> 
> a. as we do today, built on a local developer machine
> b. from Travis
> c. from buildbot
> 
> 
> https://issues.apache.org/jira/browse/INFRA-22396
> 
> 
> Ari
> 
> 
> On 10/10/21 6:38pm, Andrus Adamchik wrote:
>> Oops. I guess we shouldn't jump to GA just yet.
>> 
>>> On Oct 10, 2021, at 10:28 AM, Nikita Timofeev <nt...@objectstyle.com> wrote:
>>> 
>>> 
>>> Hi all,
>>> I've read about limitations on the Apache side. It turns out not so great in that case. There's only approved actions available and, more important, there's limited number of runners available for *all* Apache projects. Ifra warns that it could be really slow [1]
>>> [1] https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=173085013#content/view/173085013 Sunday, 10 October 2021, 10:10am +03:00 from Andrus Adamchik  aadamchik@gmail.com :
>>> 
>>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>> 
>>>> 
>>>> Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.
>>>> 
>>>> However a bit more limited with regard to database choices.
>>>> We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).
>>>> 
>>>> Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.
>>>> 
>>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>> 
>>>> Haven't tried it, good question. I hope not and it is all straight GitHub.
>>>> 
>>>> Andrus
>>>> 
>>>> 
>>>> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis < ari@ish.com.au.INVALID> wrote:
>>>> 
>>>> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>>>>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
>>>> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
>>>> 
>>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>> 
>>>> Ari
>>>> 
>>>> 
>>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>> 
> 

Re: CI

[Aristedes Maniatis <ar...@ish.com.au.INVALID>]

Should we migrate the buildbot builder on Apache infrastructure over to 
git and get it working again? Or should we put more effort into Travis 
and just abandon the Apache builder?

Are the new limits at Travis going to work for us?

I think we'll be more able to set up different database containers at 
Travis rather than at Apache.


Are we happy with creating release artifacts:

a. as we do today, built on a local developer machine
b. from Travis
c. from buildbot


https://issues.apache.org/jira/browse/INFRA-22396


Ari


On 10/10/21 6:38pm, Andrus Adamchik wrote:
> Oops. I guess we shouldn't jump to GA just yet.
>
>> On Oct 10, 2021, at 10:28 AM, Nikita Timofeev <nt...@objectstyle.com> wrote:
>>
>>
>> Hi all,
>> I've read about limitations on the Apache side. It turns out not so great in that case. There's only approved actions available and, more important, there's limited number of runners available for *all* Apache projects. Ifra warns that it could be really slow [1]
>> [1] https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=173085013#content/view/173085013 Sunday, 10 October 2021, 10:10am +03:00 from Andrus Adamchik  aadamchik@gmail.com :
>>
>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>
>>>
>>> Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.
>>>
>>> However a bit more limited with regard to database choices.
>>> We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).
>>>
>>> Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.
>>>
>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>
>>> Haven't tried it, good question. I hope not and it is all straight GitHub.
>>>
>>> Andrus
>>>
>>>
>>> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis < ari@ish.com.au.INVALID> wrote:
>>>
>>> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>>>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
>>> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
>>>
>>> Are we limited at all within the Apache project with how we can configure Actions?
>>>
>>> Ari
>>>
>>>
>>> [1]  https://github.com/ishgroup/oncourse/actions
>>>

Re: CI

[Andrus Adamchik <aa...@gmail.com>]

Oops. I guess we shouldn't jump to GA just yet.

> On Oct 10, 2021, at 10:28 AM, Nikita Timofeev <nt...@objectstyle.com> wrote:
> 
> 
> Hi all,
> I've read about limitations on the Apache side. It turns out not so great in that case. There's only approved actions available and, more important, there's limited number of runners available for *all* Apache projects. Ifra warns that it could be really slow [1]
> [1] https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=173085013#content/view/173085013 Sunday, 10 October 2021, 10:10am +03:00 from Andrus Adamchik  aadamchik@gmail.com :
> 
>> 
>> [1]  https://github.com/ishgroup/oncourse/actions
>> 
>> 
>> Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.
>> 
>> However a bit more limited with regard to database choices.
>> We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).
>> 
>> Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.
>> 
>> Are we limited at all within the Apache project with how we can configure Actions?
>> 
>> Haven't tried it, good question. I hope not and it is all straight GitHub.
>> 
>> Andrus
>> 
>> 
>> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis < ari@ish.com.au.INVALID> wrote:
>> 
>> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
>> 
>> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
>> 
>> Are we limited at all within the Apache project with how we can configure Actions?
>> 
>> Ari
>> 
>> 
>> [1]  https://github.com/ishgroup/oncourse/actions
>> 

Re[2]: CI

[Nikita Timofeev <nt...@objectstyle.com>]

Hi all,
I've read about limitations on the Apache side. It turns out not so great in that case. There's only approved actions available and, more important, there's limited number of runners available for *all* Apache projects. Ifra warns that it could be really slow [1]
[1] https://cwiki.apache.org/confluence/plugins/servlet/mobile?contentId=173085013#content/view/173085013 Sunday, 10 October 2021, 10:10am +03:00 from Andrus Adamchik  aadamchik@gmail.com :

> 
>[1]  https://github.com/ishgroup/oncourse/actions
>
>
>Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.
>
> However a bit more limited with regard to database choices.
>We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).
>
>Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.
>
> Are we limited at all within the Apache project with how we can configure Actions?
>
>Haven't tried it, good question. I hope not and it is all straight GitHub.
>
>Andrus
>
>
> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis < ari@ish.com.au.INVALID> wrote:
>
> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
>
> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
>
> Are we limited at all within the Apache project with how we can configure Actions?
>
> Ari
>
>
> [1]  https://github.com/ishgroup/oncourse/actions
>

Re: CI

[Andrus Adamchik <aa...@gmail.com>]

> [1] https://github.com/ishgroup/oncourse/actions


Nice! FWIW Bootique is using Github Actions for its dozens of modules, including cross-DB tests. A huge improvement over Travis.

> However a bit more limited with regard to database choices.

We can run tests against any database that has a Docker image. SQLServer does. (We should also switch to Testcontainers for managing dockerized tested services, but that's a separate discussion and does not affect a CI/CD choice).

Aurora is cloud-based of course, so we'll need to program provisioning the test DB ourselves, but as long as a build agent has internet access, this should be doable.

> Are we limited at all within the Apache project with how we can configure Actions?

Haven't tried it, good question. I hope not and it is all straight GitHub.

Andrus


> On Oct 10, 2021, at 9:59 AM, Aristedes Maniatis <ar...@ish.com.au.INVALID> wrote:
> 
> On 10/10/21 5:26pm, Andrus Adamchik wrote:
>> BTW, we should probably move to Github Actions for CI/CD. Much faster than Travis.
> 
> I've got a bit of experience [1] with github actions in some of my other open source projects, and mostly they are very good. Ties in nicely with pull requests as well. However a bit more limited with regard to database choices. Ideally we'd want databases like SQL Server, AWS Aurora, etc in our test suite.
> 
> Are we limited at all within the Apache project with how we can configure Actions?
> 
> Ari
> 
> 
> [1] https://github.com/ishgroup/oncourse/actions