Travis integration tests and the KubernetesExecutor

["grantnicholas2015@u.northwestern.edu" <gr...@u.northwestern.edu>]

Hey everyone. I'm working with others on the KubernetesExecutor. As we were building out pieces of the executor, we realized it would be nice to have an integration test environment to test the bits of code involving the Kubernetes integration (specifically around crash safety, but also useful for regression purposes). 

I did some testing and found running minikube on travis is possible with minikube's `driver=None` mode, which deploys kubernetes locally instead of in a VM (see here: https://github.com/kubernetes/minikube#linux-continous-integration-with-vm-support). I spun up a minikube cluster on travis this way and it's definitely technically viable. 

The issue now is it looks like no other contrib executors have integration tests, so I'm not sure what pattern I should follow on integration testing (or even if airflow wants to accept integration tests for this). There would be relatively few tests but they would be heavyweight (ie: spin up minikube, build airflow + docker image, deploy airflow+postgres on kubernetes, run tests that spin up dags and then kill the scheduler and assert everything succeeds after enough time passes for recovery).

I'm open to ideas, just wanting to get a feel from the community before starting too much work. 

Re: Travis integration tests and the KubernetesExecutor

[Grant Nicholas <gr...@u.northwestern.edu>]

Awesome. I'll try setting it up as a separate build in the Travis matrix, I
agree that seems like a good place to put it.

On Sep 15, 2017 2:27 AM, "Bolke de Bruin" <bd...@gmail.com> wrote:

> I can see the k8s integration moving to core at a certain point in time or
> being maintained outside of it but as a "core package". Why don't you make
> the ci build ENV var dependent and make sure Travis executes it as a
> separate build or builds?
>
> Bolke
>
> Verstuurd vanaf mijn iPad
>
> > Op 14 sep. 2017 om 22:21 heeft "grantnicholas2015@u.northwestern.edu"<
> grantnicholas2015@u.northwestern.edu> het volgende geschreven:
> >
> > Hey everyone. I'm working with others on the KubernetesExecutor. As we
> were building out pieces of the executor, we realized it would be nice to
> have an integration test environment to test the bits of code involving the
> Kubernetes integration (specifically around crash safety, but also useful
> for regression purposes).
> >
> > I did some testing and found running minikube on travis is possible with
> minikube's `driver=None` mode, which deploys kubernetes locally instead of
> in a VM (see here: https://github.com/kubernetes/minikube#linux-continous-
> integration-with-vm-support). I spun up a minikube cluster on travis this
> way and it's definitely technically viable.
> >
> > The issue now is it looks like no other contrib executors have
> integration tests, so I'm not sure what pattern I should follow on
> integration testing (or even if airflow wants to accept integration tests
> for this). There would be relatively few tests but they would be
> heavyweight (ie: spin up minikube, build airflow + docker image, deploy
> airflow+postgres on kubernetes, run tests that spin up dags and then kill
> the scheduler and assert everything succeeds after enough time passes for
> recovery).
> >
> > I'm open to ideas, just wanting to get a feel from the community before
> starting too much work.
> >
> >
>

Re: Travis integration tests and the KubernetesExecutor

[Bolke de Bruin <bd...@gmail.com>]

I can see the k8s integration moving to core at a certain point in time or being maintained outside of it but as a "core package". Why don't you make the ci build ENV var dependent and make sure Travis executes it as a separate build or builds?

Bolke

Verstuurd vanaf mijn iPad

> Op 14 sep. 2017 om 22:21 heeft "grantnicholas2015@u.northwestern.edu"<gr...@u.northwestern.edu> het volgende geschreven:
> 
> Hey everyone. I'm working with others on the KubernetesExecutor. As we were building out pieces of the executor, we realized it would be nice to have an integration test environment to test the bits of code involving the Kubernetes integration (specifically around crash safety, but also useful for regression purposes). 
> 
> I did some testing and found running minikube on travis is possible with minikube's `driver=None` mode, which deploys kubernetes locally instead of in a VM (see here: https://github.com/kubernetes/minikube#linux-continous-integration-with-vm-support). I spun up a minikube cluster on travis this way and it's definitely technically viable. 
> 
> The issue now is it looks like no other contrib executors have integration tests, so I'm not sure what pattern I should follow on integration testing (or even if airflow wants to accept integration tests for this). There would be relatively few tests but they would be heavyweight (ie: spin up minikube, build airflow + docker image, deploy airflow+postgres on kubernetes, run tests that spin up dags and then kill the scheduler and assert everything succeeds after enough time passes for recovery).
> 
> I'm open to ideas, just wanting to get a feel from the community before starting too much work. 
> 
>