You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Tomasz Urbaszek <tu...@apache.org> on 2020/04/15 20:13:42 UTC
Re: [Discuss] AIP-23 Proposal "Migration out of Travis CI"

Hi all,

To decrease pressure on Travis and our frustration some CI workloads were
moved to Github Actions:
https://github.com/apache/airflow/pull/8376

Hopefully there will be no failing builds...

We stil run kube tests and backports build on Travis. Further improvements
are on the way.
Thanks to Jarek Potiuk and Daniel Imberman for help and support!

Best wishes,
Tomek

On Sat, Jan 11, 2020 at 2:50 PM Kaxil Naik <ka...@gmail.com> wrote:

> Spark is using Github actions too so shouldn't be a problem I think.
>
> On Sat, Jan 11, 2020 at 1:22 PM Tomasz Urbaszek <tu...@apache.org>
> wrote:
>
> > Yes, I thought that there was limit (the docs are little bit unclear
> > regarding this issue).
> >
> > T.
> >
> > On Sat, 11 Jan 2020 at 13:32, Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> > > Oh wow! That's fantastic! Indeed it looks like in the pricing docs. I
> am
> > > sure we discussed before that there is a limit :).
> > >
> > > J.
> > >
> > >
> > > On Sat, Jan 11, 2020 at 12:41 PM Tomasz Urbaszek <turbaszek@apache.org
> >
> > > wrote:
> > >
> > > > I have checked and Github Actions have unlimited minutes for open
> > > > repositories.
> > > >
> > > > T
> > > >
> > > > On Sat, 11 Jan 2020 at 12:27, Jarek Potiuk <Jarek.Potiuk@polidea.com
> >
> > > > wrote:
> > > >
> > > > > I am working on some last fixes to Kind improvement PR and get it
> > ready
> > > > to
> > > > > merge today - I hope :). Also looking forward to moving out of
> Travis
> > > > > finally.
> > > > >
> > > > > One more thought - we think the best way to approach it is to
> enable
> > > > Github
> > > > > Actions in parallel to Travis and keep it running in parallel to
> make
> > > > sure
> > > > > it is stable.
> > > > >
> > > > > Then we can disable Travis but we can keep it available to switch
> it
> > on
> > > > > again as needed.
> > > > >
> > > > > I think the main semi-open point we have with GA is the
> availability
> > of
> > > > > credits for GCP.  The free minutes from Github will not be enough
> for
> > > > sure,
> > > > > Tomek already runs it on the GCP account we got from Google, but we
> > > need
> > > > to
> > > > > secure the credits for the future as well. I will take on that task
> > > with
> > > > > Aizhamal but we need to keep it running for some time to understand
> > > what
> > > > > the usage might be.
> > > > >
> > > > > We can also talk to Github and other providers as well and maybe
> get
> > > some
> > > > > credit donations from other people/companies (maybe we could join
> > > 'Github
> > > > > Sponsors" project and start getting some money through that
> > > > > https://github.com/sponsors)? I think having several sources of
> > > funding
> > > > > for
> > > > > our compute resources might be a good idea. WDYT?
> > > > >
> > > > > J
> > > > >
> > > > > On Sat, Jan 11, 2020 at 11:41 AM Kaxil Naik <ka...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Yeah, that would be nice.
> > > > > >
> > > > > > On Sat, Jan 11, 2020, 10:35 Tomasz Urbaszek <
> turbaszek@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > An additional advantage of GA + self-hosted runners is the
> > ability
> > > to
> > > > > > > ssh to machine and see what is going on (for example why tests
> > > > > timeout).
> > > > > > >
> > > > > > > T.
> > > > > > >
> > > > > > > On Sat, Jan 11, 2020 at 4:23 AM Kaxil Naik <
> kaxilnaik@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Awesome Tomek, great job. Waiting for your PR eagerly :) and
> > > can't
> > > > > wait
> > > > > > > to
> > > > > > > > move out of Travis.
> > > > > > > >
> > > > > > > > On Fri, Jan 10, 2020 at 2:54 PM Tomasz Urbaszek <
> > > > > > > > tomasz.urbaszek@polidea.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > Just a small update:
> > > > > > > > > - after Jarek's changes from
> > > > > > > https://github.com/apache/airflow/pull/6516
> > > > > > > > > kubernetes
> > > > > > > > >   builds started to run (proviously there was an issue with
> > > kind)
> > > > > > > > > - I am testing self-hosted runners
> > > > > > > > >
> > > > > > > > > It seems that possible migration is getting closer ;)
> > > > > > > > >
> > > > > > > > > Bests,
> > > > > > > > > Tomek
> > > > > > > > >
> > > > > > > > > On Tue, Dec 17, 2019 at 3:28 PM Philippe Gagnon <
> > > > > > philgagnon1@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > We have been using Actions on the prestosql project for a
> > > > little
> > > > > > > while
> > > > > > > > > as a
> > > > > > > > > > Travis replacement and we like it a lot.
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 17, 2019 at 9:08 AM Jarek Potiuk <
> > > > > > > Jarek.Potiuk@polidea.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > :(
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 17, 2019 at 2:35 PM Tomasz Urbaszek <
> > > > > > > > > > > tomasz.urbaszek@polidea.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I started to play with self-hosted runner and...
> well,
> > > > > > > encountered
> > > > > > > > > > known
> > > > > > > > > > > > error:
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.community/t5/GitHub-Actions/Github-action-stuck-at-queue/td-p/38003
> > > > > > > > > > > >
> > > > > > > > > > > > It seems that GA is still maturing.
> > > > > > > > > > > >
> > > > > > > > > > > > Bests,
> > > > > > > > > > > > Tomek
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Dec 13, 2019 at 5:06 PM Jarek Potiuk <
> > > > > > > > > Jarek.Potiuk@polidea.com
> > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Yeah. All at once seems more than reasonable.
> > > > > > > > > > > > >
> > > > > > > > > > > > > J.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Dec 13, 2019 at 4:50 PM Kaxil Naik <
> > > > > > > kaxilnaik@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Agree with Daniel, let's do it all at once.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Dec 13, 2019 at 3:49 PM Daniel Imberman <
> > > > > > > > > > > > > daniel.imberman@gmail.com
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I’d rather we do it all at once and not need to
> > > > > maintain
> > > > > > > two
> > > > > > > > > > > builds.
> > > > > > > > > > > > > > > Most/all of our CI/CD is dockerized at this
> point
> > > so
> > > > > > there
> > > > > > > > > > > shouldn’t
> > > > > > > > > > > > > be a
> > > > > > > > > > > > > > > huge difference.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Dec 13, 2019 at 1:23 AM, Tomasz
> Urbaszek
> > <
> > > > > > > > > > > > > > > tomasz.urbaszek@polidea.com> wrote:
> > > > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I have to solve a problem with our kubernetes
> > test
> > > > but
> > > > > > for
> > > > > > > > your
> > > > > > > > > > > > > > information
> > > > > > > > > > > > > > > I have never experienced a flaky
> > > > > > > > > > > > > > > or timeouted build on Github Actions. Maybe I
> am
> > > > lucky
> > > > > or
> > > > > > > > maybe
> > > > > > > > > > > > there's
> > > > > > > > > > > > > > > something different.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > If we agree to move to Github Actions, would we
> > > like
> > > > to
> > > > > > > > migrate
> > > > > > > > > > > > > > > incrementally or fully?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Bests,
> > > > > > > > > > > > > > > Tomek
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Wed, Dec 11, 2019 at 1:06 AM Jarek Potiuk <
> > > > > > > > > > > > Jarek.Potiuk@polidea.com
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > No problem on that side - Tomek is using the
> > same
> > > > > > scripts
> > > > > > > > we
> > > > > > > > > > have
> > > > > > > > > > > > on
> > > > > > > > > > > > > > > Travis
> > > > > > > > > > > > > > > > (and they generally work - I think the last
> > step
> > > is
> > > > > to
> > > > > > > make
> > > > > > > > > all
> > > > > > > > > > > the
> > > > > > > > > > > > > > > > tests pass. We have still a number of
> > > dependencies
> > > > > > > between
> > > > > > > > > > tests
> > > > > > > > > > > > and
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > environmental flakiness so that some tests
> > > > > consistently
> > > > > > > > fail
> > > > > > > > > in
> > > > > > > > > > > > > Github
> > > > > > > > > > > > > > > > Actions where they did not fail in Travis.
> From
> > > > > latest
> > > > > > > try
> > > > > > > > by
> > > > > > > > > > > Tomek
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > looks like we are 1 test to go (plus some
> > > > > cleanup/setup
> > > > > > > of
> > > > > > > > > > > project
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > making sure all is stable):
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > ==== 1 failed, 4030 passed, 119 skipped, 16
> > > > warnings
> > > > > in
> > > > > > > > > > 1207.96s
> > > > > > > > > > > > > > > (0:20:07)
> > > > > > > > > > > > > > > > =====
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > We discussed with Tomek and Kamil that in
> order
> > > to
> > > > > > speed
> > > > > > > it
> > > > > > > > > up
> > > > > > > > > > we
> > > > > > > > > > > > > might
> > > > > > > > > > > > > > > > want to run our own workers on the GCP
> account
> > we
> > > > > have
> > > > > > -
> > > > > > > > just
> > > > > > > > > > to
> > > > > > > > > > > > test
> > > > > > > > > > > > > > > > quickly how much we can optimise it and I am
> > > > inclined
> > > > > > to
> > > > > > > > > agree.
> > > > > > > > > > > If
> > > > > > > > > > > > we
> > > > > > > > > > > > > > do
> > > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > this way, the transition might be rather
> fast.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > If we want to use auto-scalable GKE cluster
> as
> > we
> > > > > > > > originally
> > > > > > > > > > > > planned
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > might take more time to setup (similar setup
> to
> > > > what
> > > > > I
> > > > > > > > tried
> > > > > > > > > > with
> > > > > > > > > > > > > > > GitLab).
> > > > > > > > > > > > > > > > There we might need to use docker-in-docker
> to
> > > > build
> > > > > > the
> > > > > > > CI
> > > > > > > > > > image
> > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > latest as first step of build and then use
> that
> > > > built
> > > > > > > image
> > > > > > > > > by
> > > > > > > > > > > all
> > > > > > > > > > > > > > > > subsequent steps. But we can do it as the
> next
> > > > step -
> > > > > > > > > > optimising
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > experience.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > J.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Tue, Dec 10, 2019 at 11:34 PM Daniel
> > Imberman
> > > <
> > > > > > > > > > > > > > > > daniel.imberman@gmail.com>
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > +1 on my end as well.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > @jarek does this affect anything involving
> > > > breeze?
> > > > > Do
> > > > > > > > > GitHub
> > > > > > > > > > > > > actions
> > > > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > an easy docker/docker-compose based
> workflow?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > via Newton Mail [
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.32&pv=10.14.5&source=email_footer_2
> > > > > > > > > > > > > > > > > ]
> > > > > > > > > > > > > > > > > On Tue, Dec 10, 2019 at 5:28 AM, Ash
> > > > Berlin-Taylor
> > > > > <
> > > > > > > > > > > > ash@apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > Legal: no I don't think so.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Infra: possibly yes to get secrets in there
> > to
> > > > run
> > > > > > > things
> > > > > > > > > on
> > > > > > > > > > > our
> > > > > > > > > > > > > own
> > > > > > > > > > > > > > > > > "hardware" -
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-and-using-encrypted-secrets
> > > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-and-using-encrypted-secrets
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > needs someone with Admin rights to create,
> > and
> > > I
> > > > > > don't
> > > > > > > > see
> > > > > > > > > > the
> > > > > > > > > > > > > > Settings
> > > > > > > > > > > > > > > > tab
> > > > > > > > > > > > > > > > > at all.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > -ash
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On 10 Dec 2019, at 02:46, Deng Xiaodong <
> > > > > > > > > > xd.deng.r@gmail.com
> > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > +1 for GitHub Actions. I have been using
> it
> > > for
> > > > > > > months
> > > > > > > > > for
> > > > > > > > > > my
> > > > > > > > > > > > > side
> > > > > > > > > > > > > > > > > > projects, and it’s working very well. I
> > > believe
> > > > > > most
> > > > > > > of
> > > > > > > > > us
> > > > > > > > > > > are
> > > > > > > > > > > > > > quite
> > > > > > > > > > > > > > > > > tired
> > > > > > > > > > > > > > > > > > of the waiting time using Travis CI.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > The only point I would like to remind is
> > > > whether
> > > > > we
> > > > > > > > need
> > > > > > > > > to
> > > > > > > > > > > > > > > communicate
> > > > > > > > > > > > > > > > > > with Infra/Legal team for this.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > XD
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Tue, Dec 10, 2019 at 06:49 Kaxil Naik
> <
> > > > > > > > > > > kaxilnaik@gmail.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >> +1 for Github actions
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > >> On Mon, Dec 9, 2019, 22:16 Ash
> > > Berlin-Taylor <
> > > > > > > > > > > ash@apache.org>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > >>> Happy with any thing that gives a more
> > > > seamless
> > > > > > CI
> > > > > > > > > > > > experience -
> > > > > > > > > > > > > > > > faster
> > > > > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > > > >>> good too!
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>> -a
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>> On 9 December 2019 22:12:05 GMT,
> Aizhamal
> > > > > > Nurmamat
> > > > > > > > > kyzy <
> > > > > > > > > > > > > > > > > >>> aizhamal@apache.org> wrote:
> > > > > > > > > > > > > > > > > >>>> +1 on GitHub Actions.
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>> On Mon, Dec 9, 2019 at 2:10 PM Jarek
> > > Potiuk
> > > > <
> > > > > > > > > > > > > > > > Jarek.Potiuk@polidea.com
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>>> I am all for it! GitLab has been
> > > less-than
> > > > > > > helpful
> > > > > > > > so
> > > > > > > > > > far
> > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > >>>> recently it
> > > > > > > > > > > > > > > > > >>>>> seems that running PRs from forks
> will
> > > only
> > > > > be
> > > > > > > run
> > > > > > > > in
> > > > > > > > > > > > > > Enterrprise
> > > > > > > > > > > > > > > > > >>>> Edition,
> > > > > > > > > > > > > > > > > >>>>> which is less than welcome. I am
> quite
> > a
> > > > bit
> > > > > > > > > > disappointed
> > > > > > > > > > > > > with
> > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > >>>> pace and
> > > > > > > > > > > > > > > > > >>>>> attitude. Github Actions seems to be
> > much
> > > > > > better
> > > > > > > > > > choice -
> > > > > > > > > > > > > > > > especially
> > > > > > > > > > > > > > > > > >>>> that
> > > > > > > > > > > > > > > > > >>>>> they are closely integrated with
> Github
> > > > repo
> > > > > > and
> > > > > > > > seem
> > > > > > > > > > to
> > > > > > > > > > > > get
> > > > > > > > > > > > > > > > > >>>>> attention/focus from
> Github/Microsoft.
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>> And they added self-hosted runners as
> > > well,
> > > > > > which
> > > > > > > > > makes
> > > > > > > > > > > it
> > > > > > > > > > > > > > > possible
> > > > > > > > > > > > > > > > > >>>> for us
> > > > > > > > > > > > > > > > > >>>>> to optimise the experience.
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>> J.
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>> J.
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>> On Mon, Dec 9, 2019 at 10:57 PM
> Tomasz
> > > > > > Urbaszek <
> > > > > > > > > > > > > > > > > >>>>> tomasz.urbaszek@polidea.com>
> > > > > > > > > > > > > > > > > >>>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>>> Hi all,
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> It sometime since we last discussed
> > > using
> > > > > > other
> > > > > > > CI
> > > > > > > > > > than
> > > > > > > > > > > > > > Travis.
> > > > > > > > > > > > > > > > One
> > > > > > > > > > > > > > > > > >>>> of
> > > > > > > > > > > > > > > > > >>>>> the
> > > > > > > > > > > > > > > > > >>>>>> main reasons behind considering
> Gitlab
> > > CI
> > > > > was
> > > > > > > its
> > > > > > > > > > > ability
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > work
> > > > > > > > > > > > > > > > > >>>> on
> > > > > > > > > > > > > > > > > >>>>>> self-hosted runner. However, over
> time
> > > of
> > > > > few
> > > > > > > long
> > > > > > > > > > weeks
> > > > > > > > > > > > > > Github
> > > > > > > > > > > > > > > > > >>>> Actions
> > > > > > > > > > > > > > > > > >>>>>> matured enough to allow using
> > > self-hosted
> > > > > > > runners!
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> Github Actions are still growing but
> > > using
> > > > > > them
> > > > > > > > have
> > > > > > > > > > few
> > > > > > > > > > > > big
> > > > > > > > > > > > > > > > > >>>> advantages:
> > > > > > > > > > > > > > > > > >>>>>> - they are Github natives
> > > > > > > > > > > > > > > > > >>>>>> - forking repo and enabling actions
> > will
> > > > run
> > > > > > CI
> > > > > > > on
> > > > > > > > > > your
> > > > > > > > > > > > fork
> > > > > > > > > > > > > > > > > >>>>> automatically
> > > > > > > > > > > > > > > > > >>>>>> - variety of actions (PR checks,
> > > > greetings,
> > > > > > etc)
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> I put together a PoC of CI in our
> > > internal
> > > > > > repo:
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > https://github.com/PolideaInternal/airflow/pull/542
> > > > > > > > > > > > > > > > > >>>>>> My impression is quite good. I like
> > > > > > information
> > > > > > > > > about
> > > > > > > > > > > > steps
> > > > > > > > > > > > > > > > > >>>> successes at
> > > > > > > > > > > > > > > > > >>>>>> the PR level (no need to go to CI to
> > > check
> > > > > > which
> > > > > > > > > step
> > > > > > > > > > > > > failed).
> > > > > > > > > > > > > > > The
> > > > > > > > > > > > > > > > > >>>> build
> > > > > > > > > > > > > > > > > >>>>>> log view is a little bit clumsy but
> it
> > > > > works.
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> Does any of you have any experience
> > with
> > > > > > Github
> > > > > > > > > > Actions?
> > > > > > > > > > > > Any
> > > > > > > > > > > > > > > > > >>>> thoughts
> > > > > > > > > > > > > > > > > >>>>>> about using it?
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> Best,
> > > > > > > > > > > > > > > > > >>>>>> Tomek
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>> On 2019/08/09 13:55:11, Jarek
> Potiuk <
> > > > > > > > > > > > > > Jarek.Potiuk@polidea.com>
> > > > > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>>> FYI: Interesting article about the
> > > > history
> > > > > > > behind
> > > > > > > > > > > > GitLabCI
> > > > > > > > > > > > > > > > > >>>> (featuring
> > > > > > > > > > > > > > > > > >>>>>>> Kamil, my friend).
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://about.gitlab.com/2019/08/08/built-in-ci-cd-version-control-secret/?fbclid=IwAR2tEfqLaDXTCd1mD6XUZMX7hGYBfZcohPtI2BP3-oK_Yk_EHIXF4zLDixk
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>> On Mon, Aug 5, 2019 at 7:14 PM
> Jarek
> > > > Potiuk
> > > > > > > > > > > > > > > > > >>>> <Ja...@polidea.com>
> > > > > > > > > > > > > > > > > >>>>>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> Some update on my GitLab
> experiences
> > > so
> > > > > far:
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> TL;DR; I think the POC has shown
> > that
> > > we
> > > > > can
> > > > > > > > > fairly
> > > > > > > > > > > > easily
> > > > > > > > > > > > > > > > > >>>> replicate
> > > > > > > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > > > > > > >>>>>>>> CI in GitLab + Kubernetes. I
> think i
> > > can
> > > > > > say -
> > > > > > > > it
> > > > > > > > > > > > > generally
> > > > > > > > > > > > > > > > > >>>> works, I
> > > > > > > > > > > > > > > > > >>>>>> can
> > > > > > > > > > > > > > > > > >>>>>>>> plug it in for master/v1-10-test
> > > builds
> > > > in
> > > > > > the
> > > > > > > > > main
> > > > > > > > > > > > > Airflow
> > > > > > > > > > > > > > > > > >>>> project
> > > > > > > > > > > > > > > > > >>>>>> for a
> > > > > > > > > > > > > > > > > >>>>>>>> few weeks to see how it is doing
> > > (while
> > > > I
> > > > > am
> > > > > > > no
> > > > > > > > > > > > holidays)
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > >>>> once we
> > > > > > > > > > > > > > > > > >>>>>> see
> > > > > > > > > > > > > > > > > >>>>>>>> it running and get the support for
> > PRs
> > > > > from
> > > > > > > > GitLab
> > > > > > > > > > we
> > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > >>>> switch to
> > > > > > > > > > > > > > > > > >>>>> it.
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> What do you think ? Should i call
> a
> > > vote
> > > > > or
> > > > > > > just
> > > > > > > > > try
> > > > > > > > > > > to
> > > > > > > > > > > > > set
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > >>>> up ?
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> Some details
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> - I manged to get full working
> > builds
> > > in
> > > > > > > > GitLabCI
> > > > > > > > > +
> > > > > > > > > > > > > > > > > >>>> kubernetes -
> > > > > > > > > > > > > > > > > >>>>>>>> without the kubernetes-specific
> > tests
> > > > yet,
> > > > > > but
> > > > > > > > > this
> > > > > > > > > > > > should
> > > > > > > > > > > > > > > > > >>>> be
> > > > > > > > > > > > > > > > > >>>>>> rather easy
> > > > > > > > > > > > > > > > > >>>>>>>> with kind (looking at it next):
> > > > > > > > > > > > > > > > > >>>>>>>> - Working example here - you can
> > take
> > > a
> > > > > look
> > > > > > > and
> > > > > > > > > > > compare
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > >>>>> UI/how
> > > > > > > > > > > > > > > > > >>>>>> it
> > > > > > > > > > > > > > > > > >>>>>>>> is to navigate, comparing to
> Travis
> > > etc:
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > >
> > > > https://gitlab.com/Jarek.Potiuk/airflow/pipelines/74625817
> > > > > > > > > > > > > > > > > >>>>>>>> - Per-job it is a bit slower than
> > > Travis
> > > > > so
> > > > > > > far
> > > > > > > > > > (still
> > > > > > > > > > > > > > > > > >>>> around 35
> > > > > > > > > > > > > > > > > >>>>>>>> minutes in total), but I plan to
> > > > optimise
> > > > > it
> > > > > > > > > > further.
> > > > > > > > > > > I
> > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > >>>> play
> > > > > > > > > > > > > > > > > >>>>>> with
> > > > > > > > > > > > > > > > > >>>>>>>> memory/cpu settings of individual
> > > > workers
> > > > > > (Got
> > > > > > > > > some
> > > > > > > > > > > > > > > > > >>>> reasonable
> > > > > > > > > > > > > > > > > >>>>>> values now),
> > > > > > > > > > > > > > > > > >>>>>>>> I can use local SSD disk as Docker
> > > > > > > > > storage/logs/etc
> > > > > > > > > > > > > > > > > >>>>>>>> - I got an approval for 72vCPU
> quota
> > > (up
> > > > > for
> > > > > > > > > initial
> > > > > > > > > > > > 24) -
> > > > > > > > > > > > > > > > > >>>> that
> > > > > > > > > > > > > > > > > >>>>>> should
> > > > > > > > > > > > > > > > > >>>>>>>> let us build 3 builds in parallel
> > > > > > > independently
> > > > > > > > > from
> > > > > > > > > > > > each
> > > > > > > > > > > > > > > > > >>>> other.
> > > > > > > > > > > > > > > > > >>>>>>>> - I managed to get Preemptible
> nodes
> > > > > working
> > > > > > > (we
> > > > > > > > > > have
> > > > > > > > > > > > > built
> > > > > > > > > > > > > > > > > >>>> in
> > > > > > > > > > > > > > > > > >>>>> retry
> > > > > > > > > > > > > > > > > >>>>>>>> mechanism in GitLab to work in
> case
> > of
> > > > > > system
> > > > > > > > > > failures
> > > > > > > > > > > > > like
> > > > > > > > > > > > > > > > > >>>> that
> > > > > > > > > > > > > > > > > >>>>>>>> - Current spending with > 120
> builds
> > > is
> > > > 40
> > > > > > > USD.
> > > > > > > > We
> > > > > > > > > > > > should
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > >>>> way
> > > > > > > > > > > > > > > > > >>>>>> below
> > > > > > > > > > > > > > > > > >>>>>>>> 500 USD/month according to my
> > > > > > > > back-of-the-envelope
> > > > > > > > > > > > > > > > > >>>> calculations.
> > > > > > > > > > > > > > > > > >>>>>> Likely
> > > > > > > > > > > > > > > > > >>>>>>>> well below
> > > > > > > > > > > > > > > > > >>>>>>>> - The current setup does not use
> GCR
> > > as
> > > > > > cache
> > > > > > > > and
> > > > > > > > > > > Kaniko
> > > > > > > > > > > > > as
> > > > > > > > > > > > > > > > > >>>> I
> > > > > > > > > > > > > > > > > >>>>>>>> originally planned. GCR would
> > require
> > > > > custom
> > > > > > > > > > > > > authentication
> > > > > > > > > > > > > > > > > >>>> (and
> > > > > > > > > > > > > > > > > >>>>>>>> easy-to-steal secrets) and Kaniko
> > does
> > > > not
> > > > > > yet
> > > > > > > > > well
> > > > > > > > > > > > handle
> > > > > > > > > > > > > > > > > >>>>>> multi-staging
> > > > > > > > > > > > > > > > > >>>>>>>> builds (cache does not work
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >
> > > https://github.com/GoogleContainerTools/kaniko/issues/682
> > > > > > > > > > > > > ).
> > > > > > > > > > > > > > > > > >>>> I
> > > > > > > > > > > > > > > > > >>>>>> updated
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI
> > > > > > > > > > > > > > > > > >>>>>> to
> > > > > > > > > > > > > > > > > >>>>>>>> reflect that.
> > > > > > > > > > > > > > > > > >>>>>>>> - We only use GCR as mirroring of
> > > > > DockerHub
> > > > > > -
> > > > > > > so
> > > > > > > > > > that
> > > > > > > > > > > we
> > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > >>>> have
> > > > > > > > > > > > > > > > > >>>>>>>> reliable downloads not depending
> on
> > > > > > > DockerHub's
> > > > > > > > > > > > stability
> > > > > > > > > > > > > > > > > >>>> (it has
> > > > > > > > > > > > > > > > > >>>>>> problems
> > > > > > > > > > > > > > > > > >>>>>>>> sometimes)
> > > > > > > > > > > > > > > > > >>>>>>>> - All in-all, it's
> GCP-independent.
> > It
> > > > > could
> > > > > > > be
> > > > > > > > > run
> > > > > > > > > > in
> > > > > > > > > > > > any
> > > > > > > > > > > > > > > > > >>>>>> Kubernetes
> > > > > > > > > > > > > > > > > >>>>>>>> cluster (some optimisations like
> > local
> > > > > > volumes
> > > > > > > > > > > mounting
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > >>>> docker
> > > > > > > > > > > > > > > > > >>>>>> engine
> > > > > > > > > > > > > > > > > >>>>>>>> might have GCP-specific
> assumptions,
> > > but
> > > > > > > should
> > > > > > > > be
> > > > > > > > > > > > > generally
> > > > > > > > > > > > > > > > > >>>>>> replicable).
> > > > > > > > > > > > > > > > > >>>>>>>> - You can take a look at the
> current
> > > > > source
> > > > > > > code
> > > > > > > > > in
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > >
> > https://github.com/potiuk/airflow/commits/test-gitlab-ci
> > > > > > > > > > > > > > > > > >>>>>>>> - There will be some updates (I
> will
> > > get
> > > > > rid
> > > > > > > of
> > > > > > > > > > custom
> > > > > > > > > > > > > > > > > >>>> builder
> > > > > > > > > > > > > > > > > >>>>>> Docker,
> > > > > > > > > > > > > > > > > >>>>>>>> simplify it a bit and implement
> > > > kubernetes
> > > > > > > > tests)
> > > > > > > > > -
> > > > > > > > > > > it's
> > > > > > > > > > > > > > > > > >>>> mostly
> > > > > > > > > > > > > > > > > >>>>> some
> > > > > > > > > > > > > > > > > >>>>>>>> cleanups + removal of
> > Travis-Specific
> > > > > > > variables
> > > > > > > > +
> > > > > > > > > > > > > gitlab.ci
> > > > > > > > > > > > > > > > > >>>> yaml
> > > > > > > > > > > > > > > > > >>>>>> with
> > > > > > > > > > > > > > > > > >>>>>>>> job definitions.
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> J.
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> On Wed, Jul 31, 2019 at 10:57 AM
> > Jarek
> > > > > > Potiuk
> > > > > > > <
> > > > > > > > > > > > > > > > > >>>>>> Jarek.Potiuk@polidea.com>
> > > > > > > > > > > > > > > > > >>>>>>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>> So GitLab already works on
> > > > automatically
> > > > > > > > running
> > > > > > > > > > > builds
> > > > > > > > > > > > > > from
> > > > > > > > > > > > > > > > > >>>> for PRs
> > > > > > > > > > > > > > > > > >>>>>> :).
> > > > > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>> Kamil got involved and will be
> out
> > > > > advocate
> > > > > > > on
> > > > > > > > > it:
> > > > > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > https://gitlab.com/gitlab-org/gitlab-ce/issues/65139
> > > > > > > > > > > > > > > > > >>>>>>>>> J.
> > > > > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>> Principal Software Engineer
> > > > > > > > > > > > > > > > > >>>>>>>>> Phone: +48660796129
> > > > > <+48%20660%20796%20129>
> > > > > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>> pt., 26 lip 2019, 18:12
> użytkownik
> > > > Jarek
> > > > > > > > Potiuk <
> > > > > > > > > > > > > > > > > >>>>>> Jarek.Potiuk@polidea.com>
> > > > > > > > > > > > > > > > > >>>>>>>>> napisał:
> > > > > > > > > > > > > > > > > >>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> Update: I added appropriate
> > comment
> > > in
> > > > > the
> > > > > > > > > GitLab
> > > > > > > > > > CI
> > > > > > > > > > > > > issue
> > > > > > > > > > > > > > > > > >>>> about
> > > > > > > > > > > > > > > > > >>>>> PRs
> > > > > > > > > > > > > > > > > >>>>>> and
> > > > > > > > > > > > > > > > > >>>>>>>>>> we are getting attention of
> Jason
> > > > Lenny
> > > > > -
> > > > > > > > > director
> > > > > > > > > > > of
> > > > > > > > > > > > > > > Product
> > > > > > > > > > > > > > > > > >>>>>> Management @
> > > > > > > > > > > > > > > > > >>>>>>>>>> GitLab. Let's hope they
> prioritise
> > > it
> > > > > > > quickly
> > > > > > > > > > > enough.
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> Speaking of potential
> > > > > > > complexity/Maintenance -
> > > > > > > > > in
> > > > > > > > > > > > order
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > >>>>> alleviate
> > > > > > > > > > > > > > > > > >>>>>> any
> > > > > > > > > > > > > > > > > >>>>>>>>>> maintenance worries, I think
> about
> > > > > setting
> > > > > > > up
> > > > > > > > > the
> > > > > > > > > > > > whole
> > > > > > > > > > > > > > > > > >>>> system on
> > > > > > > > > > > > > > > > > >>>>>> GitLab
> > > > > > > > > > > > > > > > > >>>>>>>>>> CI + GKE and running it in
> > parallel
> > > to
> > > > > > > Travis
> > > > > > > > > for
> > > > > > > > > > > > quite
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > >>>> time
> > > > > > > > > > > > > > > > > >>>>>> (even
> > > > > > > > > > > > > > > > > >>>>>>>>>> months) so that we can switch it
> > at
> > > > any
> > > > > > > time.
> > > > > > > > > Then
> > > > > > > > > > > we
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > >>>> able
> > > > > > > > > > > > > > > > > >>>>>> to tune
> > > > > > > > > > > > > > > > > >>>>>>>>>> it according to real use cases
> and
> > > > > compare
> > > > > > > the
> > > > > > > > > > > > > experience
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > >>>> both
> > > > > > > > > > > > > > > > > >>>>>> systems.
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> Also I am going for holidays in
> > two
> > > > > weeks
> > > > > > > and
> > > > > > > > I
> > > > > > > > > > will
> > > > > > > > > > > > > make
> > > > > > > > > > > > > > > > > >>>> sure that
> > > > > > > > > > > > > > > > > >>>>>>>>>> there will be someone with
> GitLab
> > +
> > > > > > > Kubernetes
> > > > > > > > > > > > > experience
> > > > > > > > > > > > > > > > > >>>> (from my
> > > > > > > > > > > > > > > > > >>>>>> company)
> > > > > > > > > > > > > > > > > >>>>>>>>>> who can take over and make sure
> > > there
> > > > > will
> > > > > > > be
> > > > > > > > no
> > > > > > > > > > > > > problems.
> > > > > > > > > > > > > > > > > >>>> However
> > > > > > > > > > > > > > > > > >>>>> I
> > > > > > > > > > > > > > > > > >>>>>> am
> > > > > > > > > > > > > > > > > >>>>>>>>>> quite confident :D nothing is
> > going
> > > to
> > > > > > > happen
> > > > > > > > > > while
> > > > > > > > > > > I
> > > > > > > > > > > > am
> > > > > > > > > > > > > > > > > >>>> away. I
> > > > > > > > > > > > > > > > > >>>>>> would also
> > > > > > > > > > > > > > > > > >>>>>>>>>> invite whoever from committers
> who
> > > > would
> > > > > > > like
> > > > > > > > to
> > > > > > > > > > > join
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > >>>> project
> > > > > > > > > > > > > > > > > >>>>> and
> > > > > > > > > > > > > > > > > >>>>>>>>>> gitlab instance (once I setup
> POC)
> > > to
> > > > > > learn
> > > > > > > > and
> > > > > > > > > > see
> > > > > > > > > > > > how
> > > > > > > > > > > > > > easy
> > > > > > > > > > > > > > > > > >>>> it is
> > > > > > > > > > > > > > > > > >>>>>> and how
> > > > > > > > > > > > > > > > > >>>>>>>>>> maintenance free it is going to
> > be.
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> J.
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> On Fri, Jul 26, 2019 at 2:56 PM
> > > Kamil
> > > > > > > Breguła
> > > > > > > > <
> > > > > > > > > > > > > > > > > >>>>>> kamil.bregula@polidea.com>
> > > > > > > > > > > > > > > > > >>>>>>>>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> GKE and its own CI will allow
> us
> > to
> > > > > solve
> > > > > > > > other
> > > > > > > > > > > > > problems
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > > > >>>>> building
> > > > > > > > > > > > > > > > > >>>>>>>>>>> and publishing documentation
> from
> > > the
> > > > > > > master
> > > > > > > > > > > branch.
> > > > > > > > > > > > > > > > > >>>> Currently,
> > > > > > > > > > > > > > > > > >>>>>>>>>>> building is done using the RTD
> > > > service.
> > > > > > > > > > > > Unfortunately,
> > > > > > > > > > > > > > our
> > > > > > > > > > > > > > > > > >>>> project
> > > > > > > > > > > > > > > > > >>>>>> is
> > > > > > > > > > > > > > > > > >>>>>>>>>>> too large and often the
> > > documentation
> > > > > is
> > > > > > > not
> > > > > > > > > > built
> > > > > > > > > > > > > > > properly.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > https://readthedocs.org/projects/airflow/builds/
> > > > > > > > > > > > > > > > > >>>>>>>>>>> We should think about another
> way
> > > to
> > > > > > build
> > > > > > > > > > > > > documentation.
> > > > > > > > > > > > > > > In
> > > > > > > > > > > > > > > > > >>>> the
> > > > > > > > > > > > > > > > > >>>>>> ideal
> > > > > > > > > > > > > > > > > >>>>>>>>>>> world, building documentation
> > > should
> > > > > use
> > > > > > > the
> > > > > > > > > same
> > > > > > > > > > > > > > > > > >>>> environment as
> > > > > > > > > > > > > > > > > >>>>>>>>>>> checking documentation on CI.
> > > Adding
> > > > > this
> > > > > > > > step
> > > > > > > > > to
> > > > > > > > > > > > > Travis
> > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > > >>>>> further
> > > > > > > > > > > > > > > > > >>>>>>>>>>> reduce our development
> > > opportunities.
> > > > > > > > > > > > > > > > > >>>>>>>>>>> Discussion on Slack about it:
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > >
> > > > > > >
> > > >
> https://apache-airflow.slack.com/archives/CJ1LVREHX/p1561756652021900
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> It is worth thinking about the
> > fact
> > > > > that
> > > > > > > our
> > > > > > > > > > > project
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > >>>> soon
> > > > > > > > > > > > > > > > > >>>>> have
> > > > > > > > > > > > > > > > > >>>>>> a
> > > > > > > > > > > > > > > > > >>>>>>>>>>> website and our documentation
> > will
> > > > also
> > > > > > be
> > > > > > > > > > > available
> > > > > > > > > > > > in
> > > > > > > > > > > > > > > many
> > > > > > > > > > > > > > > > > >>>>>>>>>>> languages. Currently, talks are
> > > > taking
> > > > > > > place
> > > > > > > > > with
> > > > > > > > > > > the
> > > > > > > > > > > > > > > design
> > > > > > > > > > > > > > > > > >>>>> studio
> > > > > > > > > > > > > > > > > >>>>>>>>>>> and developers who can make
> these
> > > > > > websites
> > > > > > > > ;-)
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/982c7baa06742ad722f2baa0db53ad99aea6c26b14b7d6d4aa522677@%3Cdev.airflow.apache.org%3E
> > > > > > > > > > > > > > > > > >>>>>>>>>>> We should provide an
> environment
> > > that
> > > > > > will
> > > > > > > > > allow
> > > > > > > > > > > you
> > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > >>>> build a
> > > > > > > > > > > > > > > > > >>>>>>>>>>> website and documentation. At
> > best,
> > > > > these
> > > > > > > > tasks
> > > > > > > > > > > > should
> > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > >>>>> combined.
> > > > > > > > > > > > > > > > > >>>>>> I
> > > > > > > > > > > > > > > > > >>>>>>>>>>> hope that we will be able to
> > > create a
> > > > > > > website
> > > > > > > > > > that
> > > > > > > > > > > > will
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > >>>> real
> > > > > > > > > > > > > > > > > >>>>>>>>>>> support for the community on
> > > current
> > > > > > > events,
> > > > > > > > so
> > > > > > > > > > it
> > > > > > > > > > > > will
> > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > >>>> updated
> > > > > > > > > > > > > > > > > >>>>>>>>>>> frequently.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> It seems to me that the project
> > > will
> > > > > > grow.
> > > > > > > If
> > > > > > > > > we
> > > > > > > > > > > now
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > >>>> problems
> > > > > > > > > > > > > > > > > >>>>>>>>>>> with Travis, then the
> > significance
> > > of
> > > > > > these
> > > > > > > > > > > problems
> > > > > > > > > > > > in
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > >>>> future
> > > > > > > > > > > > > > > > > >>>>>> can
> > > > > > > > > > > > > > > > > >>>>>>>>>>> only grow. Now we have a chance
> > to
> > > > > > provide
> > > > > > > a
> > > > > > > > > > stable
> > > > > > > > > > > > > > > > > >>>> infrastructure
> > > > > > > > > > > > > > > > > >>>>>> for
> > > > > > > > > > > > > > > > > >>>>>>>>>>> the project for a long time.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> I would like to share another
> > > > situation
> > > > > > > which
> > > > > > > > > was
> > > > > > > > > > > not
> > > > > > > > > > > > > > > > > >>>> pleasant for
> > > > > > > > > > > > > > > > > >>>>>> me.
> > > > > > > > > > > > > > > > > >>>>>>>>>>> Recently I wanted to send >10
> PR,
> > > but
> > > > > > > because
> > > > > > > > > of
> > > > > > > > > > > > > Travis,
> > > > > > > > > > > > > > I
> > > > > > > > > > > > > > > > > >>>> had to
> > > > > > > > > > > > > > > > > >>>>>> wait
> > > > > > > > > > > > > > > > > >>>>>>>>>>> for the weekend to send
> changes.
> > > If I
> > > > > > would
> > > > > > > > > send
> > > > > > > > > > my
> > > > > > > > > > > > > > changes
> > > > > > > > > > > > > > > > > >>>> in a
> > > > > > > > > > > > > > > > > >>>>>> week,
> > > > > > > > > > > > > > > > > >>>>>>>>>>> I would block the queue for a
> few
> > > > > hours.
> > > > > > > > > > Although I
> > > > > > > > > > > > did
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > > >>>> over
> > > > > > > > > > > > > > > > > >>>>> the
> > > > > > > > > > > > > > > > > >>>>>>>>>>> weekend, I got the message that
> > the
> > > > > queue
> > > > > > > is
> > > > > > > > > > > blocked
> > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > >>>> Travis by
> > > > > > > > > > > > > > > > > >>>>> my
> > > > > > > > > > > > > > > > > >>>>>>>>>>> jobs.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> On Tue, Jul 23, 2019 at 6:12 PM
> > > Jarek
> > > > > > > Potiuk
> > > > > > > > <
> > > > > > > > > > > > > > > > > >>>>>> Jarek.Potiuk@polidea.com>
> > > > > > > > > > > > > > > > > >>>>>>>>>>> wrote:
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> Hello Everyone,
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> I prepared a short docs where
> I
> > > > > > described
> > > > > > > > > > general
> > > > > > > > > > > > > > > > > >>>> architecture
> > > > > > > > > > > > > > > > > >>>>> of
> > > > > > > > > > > > > > > > > >>>>>> the
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> solution I imagine we can
> deploy
> > > > > fairly
> > > > > > > > > quickly
> > > > > > > > > > -
> > > > > > > > > > > > > having
> > > > > > > > > > > > > > > > > >>>> GitLab
> > > > > > > > > > > > > > > > > >>>>> CI
> > > > > > > > > > > > > > > > > >>>>>>>>>>> support
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> and Google provided funding
> for
> > > GCP
> > > > > > > > resources.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> I am going to start working on
> > > > > > > > > Proof-Of-Concept
> > > > > > > > > > > soon
> > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > >>>> before
> > > > > > > > > > > > > > > > > >>>>> I
> > > > > > > > > > > > > > > > > >>>>>>>>>>> start
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> doing it, I would like to get
> > some
> > > > > > > comments
> > > > > > > > > and
> > > > > > > > > > > > > opinions
> > > > > > > > > > > > > > > > > >>>> on the
> > > > > > > > > > > > > > > > > >>>>>>>>>>> proposed
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> approach. I discussed the
> basic
> > > > > approach
> > > > > > > > with
> > > > > > > > > my
> > > > > > > > > > > > > friend
> > > > > > > > > > > > > > > > > >>>> Kamil
> > > > > > > > > > > > > > > > > >>>>> who
> > > > > > > > > > > > > > > > > >>>>>>>>>>> works at
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> GitLab and he is a CI
> maintainer
> > > and
> > > > > > this
> > > > > > > is
> > > > > > > > > > what
> > > > > > > > > > > we
> > > > > > > > > > > > > > think
> > > > > > > > > > > > > > > > > >>>> will
> > > > > > > > > > > > > > > > > >>>>> be
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> achievable in fairly short
> time.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>
> > > > > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> I am happy to discuss details
> > and
> > > > make
> > > > > > > > changes
> > > > > > > > > > to
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > >>>> proposal -
> > > > > > > > > > > > > > > > > >>>>>> we
> > > > > > > > > > > > > > > > > >>>>>>>>>>> can
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> discuss it here or as comments
> > in
> > > > the
> > > > > > > > > document.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> Let's see what people think
> > about
> > > it
> > > > > and
> > > > > > > if
> > > > > > > > we
> > > > > > > > > > get
> > > > > > > > > > > > to
> > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > >>>>>> consensus
> > > > > > > > > > > > > > > > > >>>>>>>>>>> we
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> might want to cast a vote (or
> > > maybe
> > > > go
> > > > > > via
> > > > > > > > > lasy
> > > > > > > > > > > > > > consensus
> > > > > > > > > > > > > > > > > >>>> as
> > > > > > > > > > > > > > > > > >>>>> this
> > > > > > > > > > > > > > > > > >>>>>> is
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> something we should have
> rather
> > > > > quickly)
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> Looking forward to your
> > comments!
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> J.
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> --
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> Jarek Potiuk
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> Polidea <
> > https://www.polidea.com/
> > > >
> > > > |
> > > > > > > > > Principal
> > > > > > > > > > > > > Software
> > > > > > > > > > > > > > > > > >>>>> Engineer
> > > > > > > > > > > > > > > > > >>>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> M: +48 660 796 129
> > > > > > <+48%20660%20796%20129>
> > > > > > > > > > > > > <+48660796129
> > > > > > > > > > > > > > > > > >>>>> <+48%20660%20796%20129>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>> [image: Polidea] <
> > > > > > > https://www.polidea.com/>
> > > > > > > > > > > > > > > > > >>>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> --
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> Jarek Potiuk
> > > > > > > > > > > > > > > > > >>>>>>>>>> Polidea <
> https://www.polidea.com/
> > >
> > > |
> > > > > > > > Principal
> > > > > > > > > > > > Software
> > > > > > > > > > > > > > > > > >>>> Engineer
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> M: +48 660 796 129
> > > > > <+48%20660%20796%20129>
> > > > > > > > > > > > <+48660796129
> > > > > > > > > > > > > > > > > >>>>> <+48%20660%20796%20129>>
> > > > > > > > > > > > > > > > > >>>>>>>>>> [image: Polidea] <
> > > > > > https://www.polidea.com/>
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> --
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> Jarek Potiuk
> > > > > > > > > > > > > > > > > >>>>>>>> Polidea <https://www.polidea.com/
> >
> > |
> > > > > > > Principal
> > > > > > > > > > > Software
> > > > > > > > > > > > > > > > > >>>> Engineer
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>> M: +48 660 796 129
> > > > <+48%20660%20796%20129>
> > > > > > > > > > > <+48660796129
> > > > > > > > > > > > > > > > > >>>>> <+48%20660%20796%20129>>
> > > > > > > > > > > > > > > > > >>>>>>>> [image: Polidea] <
> > > > > https://www.polidea.com/>
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>> --
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>> Jarek Potiuk
> > > > > > > > > > > > > > > > > >>>>>>> Polidea <https://www.polidea.com/>
> |
> > > > > > Principal
> > > > > > > > > > > Software
> > > > > > > > > > > > > > > Engineer
> > > > > > > > > > > > > > > > > >>>>>>>
> > > > > > > > > > > > > > > > > >>>>>>> M: +48 660 796 129
> > > > <+48%20660%20796%20129>
> > > > > > > > > > > <+48660796129
> > > > > > > > > > > > > > > > > >>>>> <+
> >
>