You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/12 18:05:51 UTC
[GitHub] [airflow] potiuk opened a new pull request #8265: Kubernetes on host
potiuk opened a new pull request #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8265: Kubernetes on host
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612993074
The following Kubernetes versions are currently available for new clusters in Amazon EKS:
* 1.15.11
* 1.14.9
* 1.13.12
* 1.12.10
You can use the newer version if you use GKE on AWS. However, this is a new service that was released yesterday.
https://cloud.google.com/anthos/gke/docs/aws/concepts/architecture
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8265: Kubernetes on host
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-613056910
No reason whatsoever. Just carried from the past. Right now it's as easy as modifying a few constants :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8265: Kubernetes on
host
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612993074
The following Kubernetes versions are currently available for new clusters in Amazon EKS:
* 1.15.11
* 1.14.9
* 1.13.12
* 1.12.10
https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html
You can use the newer version if you use GKE on AWS. However, this is a new service that was released yesterday.
https://cloud.google.com/anthos/gke/docs/aws/concepts/architecture
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on issue #8265: Kubernetes on host
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612978171
@potiuk is there a reason we're still defaulting to such an old k8s? We should set it to at least 1.17 (unless that should be a separate PR).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #8265: Kubernetes on
host
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612993074
The following Kubernetes versions are currently available for new clusters in Amazon EKS:
* 1.15.11
* 1.14.9
* 1.13.12
* 1.12.10
You can use the newer version if you use GKE on AWS. However, this is a new service that was released yesterday.
https://cloud.google.com/anthos/gke/docs/aws/concepts/architecture
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8265: Kubernetes on host
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612655321
Hey @dimberman @kaxil @turbaszek -> I have almost working change where Kind Kubernetes cluster is created on the host side and it can be used to run the tests with Kubernetes. I nicely integrated it with breeze so that you can run:
`breeze kind-cluster start` -> will start the kind cluster
`breeze kind-cluster deploy --force-build` -> will deploy airflow to the cluster
and then you should be able to run tests. I just wanted to share it with you so that you can take a look. This can be very easily reused for the helm chart deployments (@dimbermann) and I think it's a better way of handling Kind Setup than relying in Github Action specific configuration from https://github.com/apache/airflow/pull/8222 - because it can be run for local testing as well and it can be run on any CI (Travis/GA/Cloud Build. ....) . I updated the documentation including explanation how I imagine running the tests locally with breeze.
I am planning to complete it tomorrow - but please take a look in the meantime :).
It is based on the #8261 so only look at the latest commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #8265: Kubernetes on host
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #8265: Kubernetes on host
URL: https://github.com/apache/airflow/pull/8265#issuecomment-612989890
@dimberman Old release? Version 1.14 is the latest version of the stable release channel on GKE. Version 1.15 is the latest version of the regular release channel. I don't think we need to use the latest possible version, but the most popular version.
https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels#finding_out_whats_new
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services