[jira] [Comment Edited] (AIRFLOW-6435) Mount secret files from secrets config

["Brandon Clark (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007248#comment-17007248 ] 

Brandon Clark edited comment on AIRFLOW-6435 at 1/3/20 6:15 AM:
----------------------------------------------------------------

I have created an implementation which requires the following entry in airflow.cfg which will reference a Kubernetes secret configuration for keys matching each filename and place them in the corresponding folder:
{code:java}
secret_file_secret = airflow
secret_file_dir = /root/.ssh
secret_file_filenames = airflow.pem,client.key{code}
 

The generated output results in the following additions to a pod configuration:
{code:java}
apiVersion: v1
kind: Pod
metadata:
  name: example-dag
spec:
  containers:
    volumeMounts:
    - mountPath: /root/.ssh/airflow.pem
      name: airflow-secret-files
      subPath: airflow.pem
    - mountPath: /root/.ssh/client.key
      name: airflow-secret-files
      subPath: client.key
  volumes:
  - name: airflow-secret-files
    secret:
      defaultMode: 256
      items:
      - key: airflow.pem
        path: airflow.pem
      - key: client.key
        path: client.key
      secretName: airflow{code}
 

If this seems acceptable I can make a commit to 1.10.x and 2.0.x code bases.  Any suggestions before I attempt to make a commit request?


was (Author: webmind):
I have created an implementation which requires the following entry in airflow.cfg which will reference a Kubernetes secret configuration for keys matching each filename and place them in the corresponding folder:
{code:java}
    secret_file_filenames = airflow.pem,client.key
    secret_file_secret = airflow
    secret_file_dir = /root/.ssh {code}
 

The generated output results in the following additions to a pod configuration:
{code:java}
apiVersion: v1
kind: Pod
metadata:
  name: example-dag
spec:
  containers:
    volumeMounts:
    - mountPath: /root/.ssh/airflow.pem
      name: airflow-secret-files
      subPath: airflow.pem
    - mountPath: /root/.ssh/client.key
      name: airflow-secret-files
      subPath: client.key
  volumes:
  - name: airflow-secret-files
    secret:
      defaultMode: 256
      items:
      - key: airflow.pem
        path: airflow.pem
      - key: client.key
        path: client.key
      secretName: airflow{code}
 

If this seems acceptable I can make a commit to 1.10.x and 2.0.x code bases.  Any suggestions before I attempt to make a commit request?

> Mount secret files from secrets config
> --------------------------------------
>
>                 Key: AIRFLOW-6435
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6435
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: executor-kubernetes
>    Affects Versions: 2.0.0, 1.10.8
>            Reporter: Brandon Clark
>            Assignee: Brandon Clark
>            Priority: Major
>              Labels: Kubernetes, executor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> There should be a dynamic way to add protected files to pod.  Just as git-sync requires an ssh key to mounted so do other software suites and processes that can be ran from Airflow.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)