You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Kevin Pullin (JIRA)" <ji...@apache.org> on 2018/11/26 22:02:00 UTC
[jira] [Commented] (AIRFLOW-3402) Set default kubernetes affinity
and toleration settings in airflow.cfg
[ https://issues.apache.org/jira/browse/AIRFLOW-3402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16699621#comment-16699621 ]
Kevin Pullin commented on AIRFLOW-3402:
---------------------------------------
A working example of this is here: [https://github.com/kppullin/incubator-airflow/commit/b60b0f2760e76e3f09372d2117a62e2df59dc717] . You'll note that in the airflow.cfg file, the affinity and toleration lines are defined as a single line of JSON data. The object model for these configs are complex and hard to express in the existing INI style config file. I'm of course open to suggestions.
> Set default kubernetes affinity and toleration settings in airflow.cfg
> ----------------------------------------------------------------------
>
> Key: AIRFLOW-3402
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3402
> Project: Apache Airflow
> Issue Type: Improvement
> Components: kubernetes
> Reporter: Kevin Pullin
> Priority: Major
>
> Currently airflow supports setting kubernetes `affinity` and `toleration` configuration inside dags using either a `KubernetesExecutorConfig` definition or using the `KubernetesPodOperator`.
> In order to reduce having to set and maintain this configuration in every dag, it'd be useful to have the ability to set these globally in the airflow.cfg file. One use case is to force all kubernetes pods to run on a particular set of dedicated airflow nodes, which requires both affinity rules and tolerations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)