Fwd: [DISCUSSION] Use github actions for python wheels ?

[Austin Bennett <wh...@gmail.com>]

There seems to be support here for this idea.  After digging through
things, I *think* I understand the moving pieces and can address (reason I
was digging through the code, including beam-wheels repo [1]).  Though, not
100% that I have not overlooked bits.

Would be happy to pickup
https://issues.apache.org/jira/browse/BEAM-9388 (imagine
would get to it over the course of a month or few -- so also wouldn't want
to take it on if others would be looking to do it sooner) -- seems
worthwhile to reduce manual steps for the release process, as well as the
additional implications (daily builds) if can easily manage.  I have some
experience with GitHub Actions, but not with our (BEAM) deployment/release
process, nor much with building wheels.  I assume there are sufficient docs
available to go through for the mechanics of the latter.  If picking up
this issue, I suspect may need to bother some of those that have done
recent releases -- that would best occur on the jira ticket, once at that
point?

@Ismail <ie...@gmail.com>-- given your experience with GitHub actions,
any reason to think GitHub Actions not appropriate/ready for this specific
task?

@Ahmet Altay <al...@google.com> happy to understand the extent of what you
had in mind, maybe the extensions are not as important to plan out, as
they're straightforwardly bolted on (ex: daily builds).  More tactically
would be valuable to ensure I understand what all needs to occur.  Any
other source of info to consume other than
https://github.com/apache/beam-wheels and
https://beam.apache.org/contribute/release-guide/.

Also, open to the thought that this might be taking on too much, without
more experience with the release process and such.  Do advise...?


[1]
https://lists.apache.org/thread.html/r93884eb080297647207f7d2b8a393e224029fc2c3509017886e84051%40%3Cdev.beam.apache.org%3E



---------- Forwarded message ---------
From: Ismaël Mejía <ie...@gmail.com>
Date: Wed, Feb 26, 2020 at 12:09 PM
Subject: Re: [DISCUSSION] Use github actions for python wheels ?
To: dev <de...@beam.apache.org>


+1 I have been migrating multiple projects into github actions recently and
even if for some tasks it is not as mature and polished as travis it has
proven to be way more reliable.


On Wed, Feb 26, 2020 at 7:07 PM Ahmet Altay <al...@google.com> wrote:

> I created https://issues.apache.org/jira/browse/BEAM-9388 to explore
> this. To be explicit and not to do cookie licking, I would not be able to
> work on this at the moment. If anyone is interested please take it.
> Otherwise I will try to come back and explore this when I can.
>
> On Tue, Feb 25, 2020 at 2:57 PM Robert Bradshaw <ro...@google.com>
> wrote:
>
>> I'd be in favor of this, assuming it actually simplifies things.
>
>
> This is also my concern. I do think that it will simplify things, but I am
> not certain as I am not very familiar with the github actions.
>
>
>> (Note
>> that the wheels are for several variants of linux, presumably we could
>> do cross-compiles. Also, manylinux is a "minimal" linux specifically
>> built as to produce shared object libraries compatible with a wide
>> variety of distributions--we can't just assume that a shared object
>> library built on one modern linux will just work on another. (But
>> maybe it's sufficient to do this within a docker environment?)
>>
>
> There will be no change in this area. Both in Both Travis and github
> actions offer a comparable set of options.
>
>
>>
>> On Tue, Feb 25, 2020 at 2:23 PM Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > +1 to exploring this.
>> >
>> > On builds@apache.org there is lots of discussion and general approval
>> for trying it. It is enabled and used by some projects. Calcite uses it to
>> build their website, for example.
>>
>
> Great.
>
>
>> >
>> > Kenn
>> >
>> >
>> > On Tue, Feb 25, 2020 at 2:08 PM Ahmet Altay <al...@google.com> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> I recently had a chance to look at the documentation for github
>> actions. I think we could use github actions instead of travis to for
>> building python wheels during releases. This will have the following
>> advantages:
>> >>
>> >> - We will eliminate one repo. (If you don't know, we have
>> https://github.com/apache/beam-wheels for the sole purpose of building
>> wheels file.)
>> >> - Workflow will be stored in the same repo. This will prevent bit rot
>> that is only discovered at release times. (happened a few times, although
>> usually easy to fix.)
>> >> - github actions supports ubuntu, mac, windows environments. We could
>> try to build wheels for windows as well. (Travis also supports the same
>> environments but we only use linux and mac environments. Maybe there are
>> other blockers for building wheels for Windows.)
>> >> - We could do more, like daily python builds.
>> >>
>> >> Downsides would be:
>> >> - I do not know if github actions will require some special set of
>> permissions that require an approval from infra.
>> >> - Travis works fine most of the time. This might be unnecessary work.
>> >>
>> >> What do you think? Is this feasible, would this be useful?
>> >>
>> >> Ahmet
>>
>