[jira] [Created] (AIRFLOW-4385) Add namespace into pod's yaml

["Nikolay (JIRA)" <ji...@apache.org>]

Nikolay created AIRFLOW-4385:
--------------------------------

             Summary: Add namespace into pod's yaml
                 Key: AIRFLOW-4385
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4385
             Project: Apache Airflow
          Issue Type: Improvement
          Components: kubernetes, worker
            Reporter: Nikolay
            Assignee: Nikolay
         Attachments: pods_logs.txt

During integration phase between Airflow and VerticalPodAutoscaler (link: [https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler]) i stumbled into an issue when VerticalPodAutoscaler library doesn't work for Airflow worker pods. 

How VerticalPodAutoscaler produces resource recommendations:
- monitor newly created pods
- compare newly created pods with existing configuration (compare by label and namespace)
- provide new recommendations if configuration exists for pod

 VerticalPodAutoscaler finds all created pods by label, but namespace comparison fails every time. I did some investigation and realise that when airflow calls a kubernetes api, it passes yaml file without +metadata.namespace+ property. Go libraries can deserialise this yaml, but there is no information about namespace.

Attaching logs from VerticalPodAutoscaler, there you can see how airflow pod was deserialised (namespace is missing). Works for custom pod.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)