Github Actions for CI

[Francis Chuang <fr...@apache.org>]

A month or so ago, I started a thread on Github Actions on the list. I 
have just replaced Travis with Github Actions for Avatica-Go (CALCITE-3356).

For the workflows in Avatica-Go (currently just 1 for CI), see: 
https://github.com/apache/calcite-avatica-go/tree/master/.github/workflows

The latest run is here: 
https://github.com/apache/calcite-avatica-go/runs/226087805

The VMs provided by Github are a lot beefier (2 core CPUs, 7GB RAM, 14GB 
SSD disk). The tests are slightly slower compare to Travis (in the order 
of seconds), but this is most likely due to us having to check out the 
code during run time and the need to compile the Github Action for 
checking out the code for each run. I don't think this is too much of an 
issue and the direct integration with Github is much nicer.

I think it shouldn't be too difficult to move Avatica and Calcite over. 
We can probably get rid of Appveyor as well since Github Actions 
provides Windows and OS X nodes as well.

One thing that would be interesting would be to move the integration 
tests for Calcite to docker images and run each test for each backend in 
its own job to maximize parallelism and allow each job to maximize the 
available hardware resources.

Francis

Re: Github Actions for CI

[Vladimir Sitnikov <si...@gmail.com>]

>Definitely would simplify some of
>the RM steps for a release.

I've recently added GitHub Actions config to test Apache JMeter for Windows
and macOS.
So far I'm impressed.

Pros:
* It starts quite fast. Appveyor might take 2-3hours to even start the
build.
Actions start very fast, and it does catch Windows-specific issues like
CRLF, "un-ability to remove a file that is not closed" and "un-ability to
remove a read-only file"

Cons:
* It always fetches all the refs from the Git repository (not just
master+pr, but it fetches even gh-pages branch) which causes issues like
https://github.com/junit-team/junit5/issues/2048 . It is fixable, but the
defaults are odd.
* Caching seems to be not there. Dependencies seem to be downloaded on each
build.
* There's a lot of jitter. JMeter does have several tests like
"thread.sleep(200) + assert actual duration", and they fail way more often
in Actions.
* GitHub Actions is likely unavailable for the forked repositories. In
other words, I cannot easily use Actions in my fork repository. I can use
Travis, and Calcite's travis.yml works for my repository. However, if
Calcite migrates to Actions, then forks would be harder to test (PR would
be required).

Vladimir

Re: Github Actions for CI

[Kevin Risden <kr...@apache.org>]

So I've been wanting to sit down and look at this closer so its exciting to
see progress :) Thanks for digging in. I'd be interested in looking at
replacing Travis and seeing what it takes for the build with separate JVMs.

One thing that would be interesting would be to move the integration
> tests for Calcite to docker images and run each test for each backend in
> its own job to maximize parallelism and allow each job to maximize the
> available hardware resources.
>

I think this is a separate but related idea. It would be good to have
integration tests run without a beefy VM. Definitely would simplify some of
the RM steps for a release.

Kevin Risden


On Tue, Sep 17, 2019 at 7:20 PM Francis Chuang <fr...@apache.org>
wrote:

> A month or so ago, I started a thread on Github Actions on the list. I
> have just replaced Travis with Github Actions for Avatica-Go
> (CALCITE-3356).
>
> For the workflows in Avatica-Go (currently just 1 for CI), see:
> https://github.com/apache/calcite-avatica-go/tree/master/.github/workflows
>
> The latest run is here:
> https://github.com/apache/calcite-avatica-go/runs/226087805
>
> The VMs provided by Github are a lot beefier (2 core CPUs, 7GB RAM, 14GB
> SSD disk). The tests are slightly slower compare to Travis (in the order
> of seconds), but this is most likely due to us having to check out the
> code during run time and the need to compile the Github Action for
> checking out the code for each run. I don't think this is too much of an
> issue and the direct integration with Github is much nicer.
>
> I think it shouldn't be too difficult to move Avatica and Calcite over.
> We can probably get rid of Appveyor as well since Github Actions
> provides Windows and OS X nodes as well.
>
> One thing that would be interesting would be to move the integration
> tests for Calcite to docker images and run each test for each backend in
> its own job to maximize parallelism and allow each job to maximize the
> available hardware resources.
>
> Francis
>