You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/05 14:12:01 UTC
[GitHub] [airflow] micke-a opened a new issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
micke-a opened a new issue #17447:
URL: https://github.com/apache/airflow/issues/17447
**Description**
Option to exclude default / hard-coded helm annotations for jobs and other relevant templates.
**Use case / motivation**
Ability to exclude default helm annotations will make it easier to use tools such as ArgoCD and its Helm integration to manage deployments.
ArgoCD has some understanding of Helm annotations [ArgoCD docs](https://argo-cd.readthedocs.io/en/stable/user-guide/helm/#helm-hooks) but wants us to override certain things like "hook-weight" to make sure the resource is created at the right time.
Currently it is not possible to avoid the default annotations being created and we have tested to add-on the ArgoCD specific annotations but they don't override the default ones.
Which results in ArgoCD deployments not working due to e.g. deployments waiting forever for the db migration job to complete (which doesn't run).
**Are you willing to submit a PR?**
Possibly, new to Helm and unsure how I can actually test it out : )
Not been able to test this yet but started to work on a potential solution to this; adding `defaultJobAnnotations: true` configuration attribute.
Example:
```
config:
migrateDatabaseJob:
defaultJobAnnotations: false
jobAnnotations:
"argocd.argoproj.io/hook": PreSync
createUserJob:
defaultJobAnnotations: false
jobAnnotations:
"argocd.argoproj.io/hook": PreSync
```
Which could be used like this in the template:
```
annotations:
{{- if .Values.migrateDatabaseJob.defaultJobAnnotations }}
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook-weight": "1"
"helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
{{- end }}
{{- if .Values.migrateDatabaseJob.jobAnnotations }}
{{- toYaml .Values.migrateDatabaseJob.jobAnnotations | nindent 4 }}
{{- end }}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] burakovsky commented on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
burakovsky commented on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-989374372
Is it possible to add a similar configuration for other resources with hardcoded helm annotations (extra-configmaps, redis-secrets.yaml) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-1004536503
This is supported now that #18776 has been merged.
@burakovsky, that is something I've been pondering as well. I'll try and take a look here soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] burakovsky edited a comment on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
burakovsky edited a comment on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-989374372
Is it possible to add a similar configuration for other resources with hardcoded helm annotations (extra-configmaps, redis-secrets.yaml) ? Are these annotations really required in configmaps and secrets?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-893493189
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham closed issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
jedcunningham closed issue #17447:
URL: https://github.com/apache/airflow/issues/17447
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] angel9484 commented on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
angel9484 commented on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-917575251
Coming from #18172
Good idea!
I think that beyond to solve what technology is behind the behavior of the object scheduling order (argocd, vanilla helm...), the wait-for-migrations initContainer should be removed and separate the responsabilities of the migration job and the correct startup of the airflow parts. Is a solution more complex compared yours but I think is better in a long term
Kind regards!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] burakovsky edited a comment on issue #17447: Option to exclude default / hard-coded helm annotations for jobs and other relevant templates
Posted by GitBox <gi...@apache.org>.
burakovsky edited a comment on issue #17447:
URL: https://github.com/apache/airflow/issues/17447#issuecomment-989374372
Is it possible to add a similar configuration for other resources with hardcoded helm annotations (extra-configmaps, redis-secrets.yaml) ? Are these annotations really required in configmaps and secrets as Helm creates it [before](https://github.com/helm/helm/blob/main/pkg/releaseutil/kind_sorter.go#L39-L41) most other resources?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org