You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/23 10:56:48 UTC
[GitHub] [airflow] jaydesl opened a new pull request #11769: Update charts to follow Helm RBAC best practices
jaydesl opened a new pull request #11769:
URL: https://github.com/apache/airflow/pull/11769
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
Update the Helm charts to follow best practices around RBAC (as suggested in #11755) . Namely:
* `rbac.create` replaces `rbacEnabled`
* ServiceAccount creation can be enabled individually with `...serviceAccount.create`
* ServiceAccounts can be named with `...serviceAccount.name` - if not given, previous naming conventions are used
* WebServer SA creation is now conditional (previously it was created even if `rbacEnabled = False`)
closes: #11755
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-808599188
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DerekHeldtWerle edited a comment on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
DerekHeldtWerle edited a comment on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-769299624
@mik-laj @jaydesl, I've rebased this PR with master [here](https://github.com/DerekHeldtWerle/airflow/tree/fix/rbac) and added all of the additional requirements needed to get this chart deployed when working in environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without providing any additional permssions to the `default` service account in the namespace this is deployed to.
When enabled, _every_ pod in the cluster must have a serviceaccount associated with it that is then binded to a psp. By having the option to set the serviceaccount for every potential pod allows teams to create a single service account mapped to a psp and set that for every pod. Long term, adding psp's at a per service level (e.g. airflow, flower, pgbouncer) would be the best path forward, but this is an initial step towards supporting that.
I'm happy to open up a new PR that includes this PR's changes and more, but can wait until this PR is merged and add my changes after the fact if that is deemed more appropriate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-769606653
Related: https://github.com/apache/airflow/issues/13643
@FloChehab Could you please make a review and say if the changes presented are in line with your issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DerekHeldtWerle edited a comment on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
DerekHeldtWerle edited a comment on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-769299624
@mik-laj @jaydesl, I've rebased this PR with master [here](https://github.com/DerekHeldtWerle/airflow/tree/fix/rbac) and added all of the additional requirements needed to get this chart deployed when working in environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled without having to provide any additional permissions to the `default` service account in the namespace this is deployed to.
When enabled, _every_ pod in the cluster must have a serviceaccount associated with it that is then binded to a psp. By having the option to set the serviceaccount for every potential pod allows teams to create a single service account mapped to a psp and set that for every pod. Long term, adding psp's at a per service level (e.g. airflow, flower, pgbouncer) would be the best path forward, but this is an initial step towards supporting that.
I'm happy to open up a new PR that includes this PR's changes and more, but can wait until this PR is merged and add my changes after the fact if that is deemed more appropriate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-715268725
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, itβs a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better π.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-715288886
I think it's worth adding more unit tests here to make sure everything works. Unfortunately, we are currently migrating to [new framework](https://github.com/apache/airflow/issues/11657) in which writing tests will be much easier and faster. This means that we have to wait to merge this change until we have a new framework.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-751239982
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #11769:
URL: https://github.com/apache/airflow/pull/11769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jaydesl commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
jaydesl commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-773132365
Hi @mik-laj sorry this fell off my radar. Let me know if you want me to rebase this or alternatively if its easier just go with the single PR from @DerekHeldtWerle (thanks btw!)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jaydesl commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
jaydesl commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-720323929
Sure, I'll do both of these things today.
Jay
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DerekHeldtWerle commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
DerekHeldtWerle commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-776102726
@jaydesl @FloChehab I went ahead and rebased with master again and opened up a new PR [here](https://github.com/apache/airflow/pull/14152) to keep this moving along. Let me know if there's anything on the PR that needs to be addressed and I'll get to it as soon as I can π
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-751248176
@jaydesl What is the status of this PR? Is it ready for review?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DerekHeldtWerle commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
DerekHeldtWerle commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-775310698
@FloChehab
> I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).
My PR doesn't actually add any PSP related items, all it does it create all of the necessary service accounts that are required to be set when PSP's come into play as well as rebasing with master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] FloChehab commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
FloChehab commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-775405792
> @FloChehab
>
> > I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).
>
> My PR doesn't actually add any PSP related items, all it does it create all of the necessary service accounts that are required to be set when PSP's come into play as well as rebasing with master.
Sorry I think I saw some things related to uids and I thought there might be some more complex things related to psp in your PR.
I don't really have an opinion on what is best to do. I am sure we will end up in the right spot some way or another.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DerekHeldtWerle commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
DerekHeldtWerle commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-769299624
@mik-laj @jaydesl, I've rebased this PR with master [here](https://github.com/DerekHeldtWerle/airflow/tree/fix/rbac) and added all of the additional requirements needed to get this chart deployed when working in environments where users have the [PodSecurityPolicy](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy) admission controller enabled.
When enabled, _every_ pod in the cluster must have a serviceaccount associated with it that is then binded to a psp. By having the option to set the serviceaccount for every potential pod allows teams to create a single service account mapped to a psp and set that for every pod. Long term, adding psp's at a per service level (e.g. airflow, flower, pgbouncer) would be the best path forward, but this is an initial step towards supporting that.
I'm happy to open up a new PR that includes this PR's changes and more, but can wait until this PR is merged and add my changes after the fact if that is deemed more appropriate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] FloChehab commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
FloChehab commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-774746921
> Related: #13643
>
> @FloChehab Could you please make a review and say if the changes presented are in line with your issue?
Hello @mik-laj, sorry for the response delay, I had other things on top of my mind lately.
The changes mentioned in the description of the PR looks like a step in the right direction, in particular in regards to #13643 & https://github.com/apache/airflow/issues/13643#issuecomment-774746562.
I would personally rebase this and handle the PSP related changes from @DerekHeldtWerle in a different PR (there still might some service accounts to add to this PR?).
Also, there might tests to add to make sure the annotations work properly (kind of like this: https://github.com/apache/airflow/pull/13615/files).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jaydesl commented on pull request #11769: Update charts to follow Helm RBAC best practices
Posted by GitBox <gi...@apache.org>.
jaydesl commented on pull request #11769:
URL: https://github.com/apache/airflow/pull/11769#issuecomment-773132365
Hi @mik-laj sorry this fell off my radar. Let me know if you want me to rebase this or alternatively if its easier just go with the single PR from @DerekHeldtWerle (thanks btw!)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org