Dockerised CI and testing environment

[Gerardo Curiel <ge...@gerar.do>]

Hello folks,

I just submitted a PR for using Docker as part of Airflow's build pipeline:
https://github.com/apache/incubator-airflow/pull/3393

Currently, running unit tests is a difficult process. Airflow tests depend
on many external services and other custom setup, which makes it hard for
contributors to work on this codebase. CI builds have also been unreliable,
and it is hard to reproduce the causes. Having contributors trying to
emulate the build environment every time makes it easier to get to an "it
works on my machine" sort of situation. The proposed docker setup aims to
simplify this.

You can check the PR description, which goes into more details. Now, I
bring this to the list because I have a few requests:

- Could you try the branch out on your local machines? The instructions are
provided in the PR (pending: actual docs). I would love to get feedback
about it.
- Is anyone familiar with the impersonation tests? There have proven to be
hard to fix with my limited knowledge of the codebase

It's WIP, but I wanted to submit what I've got so far and check if you guys
think I'm going in the right direction.

Cheers,

-- 
Gerardo Curiel // https://gerar.do

Re: Dockerised CI and testing environment

[Daniel Imberman <da...@gmail.com>]

Hi Gerardo,

I left some comments on the PR. Could you please get the travis tests to
pass + rebase your PR? Afterwards I'd be glad to try it out.

On Mon, May 21, 2018 at 6:55 AM Gerardo Curiel <ge...@gerar.do> wrote:

> Hello folks,
>
> I just submitted a PR for using Docker as part of Airflow's build pipeline:
> https://github.com/apache/incubator-airflow/pull/3393
>
> Currently, running unit tests is a difficult process. Airflow tests depend
> on many external services and other custom setup, which makes it hard for
> contributors to work on this codebase. CI builds have also been unreliable,
> and it is hard to reproduce the causes. Having contributors trying to
> emulate the build environment every time makes it easier to get to an "it
> works on my machine" sort of situation. The proposed docker setup aims to
> simplify this.
>
> You can check the PR description, which goes into more details. Now, I
> bring this to the list because I have a few requests:
>
> - Could you try the branch out on your local machines? The instructions are
> provided in the PR (pending: actual docs). I would love to get feedback
> about it.
> - Is anyone familiar with the impersonation tests? There have proven to be
> hard to fix with my limited knowledge of the codebase
>
> It's WIP, but I wanted to submit what I've got so far and check if you guys
> think I'm going in the right direction.
>
> Cheers,
>
> --
> Gerardo Curiel // https://gerar.do
>