[jira] [Commented] (AIRFLOW-5637) KubeExecutor - Resource Request and Limits

["Daniel Tahara (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-5637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953994#comment-16953994 ] 

Daniel Tahara commented on AIRFLOW-5637:
----------------------------------------

3819 was closed due to the recommendation to handle it on the k8s side of things. As mentioned in my comment on github, I don't actually think that's an appropriate configuration.

https://github.com/apache/airflow/pull/4660#issuecomment-542982778

```

Since LimitRanges are applied as a namespace default, the more tenants you have the less useful those ranges are going to be (because they'll have to be uselessly wide). There isn't a good reason that Airflow should be creating pods without memory limits or not providing that configuration option out of the box. It makes binpacking and cluster autoscaling much harder, since the pods will schedule effectively anywhere.

```

 

The existing proposed PR would be sufficient to solve the problem.

> KubeExecutor - Resource Request and Limits
> ------------------------------------------
>
>                 Key: AIRFLOW-5637
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5637
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: executor-kubernetes
>    Affects Versions: 1.10.5
>            Reporter: F D
>            Assignee: Daniel Imberman
>            Priority: Minor
>
> KubeExecutor doesn't have a way to have pods be created with resource requests or limits. 
> The only workaround is to set Namespace defaults, which the created pods pick up.
> I feel this could be a setting in `[kuberetes]` in `airflow.cfg`, or use the `[operator]` defaults.
> Here is the reference of the spec:
> [https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#resourcerequirements-v1-core]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)