[jira] [Created] (AIRFLOW-4986) PATH variable isn't passed down when the worker while using run_as_user in the DAG

["Aneesh Joseph (JIRA)" <ji...@apache.org>]

Aneesh Joseph created AIRFLOW-4986:
--------------------------------------

             Summary: PATH variable  isn't passed down when the worker while using run_as_user in the DAG
                 Key: AIRFLOW-4986
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4986
             Project: Apache Airflow
          Issue Type: Bug
          Components: worker
    Affects Versions: 1.10.3
            Reporter: Aneesh Joseph


I have installed airflow into a virtual environment and I start up the airflow worker using a systemd script with 
{code:java}
EnvironmentFile=/mnt/airflow/airflow.envExecStart=/bin/bash -a -c 'export AIRFLOW_HOME && export PATH && exec /mnt/airflow/venv/bin/airflow worker --queues queue1,queue2'{code}
/mnt/airflow/airflow.env looks like this

 
{code:java}
AIRFLOW_HOME=/mnt/airflow/
PATH=/mnt/airflow/venv/bin/:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin{code}
 

The service starts up ok and the worker shows up on the celery flower UI. The worker correctly picks up the DAGs scheduled on queue1 or queue2, but the tasks with a  *run_as_user* setting fail without showing much info on the Airflow UI logs, but looking at the worker log for that specific task instance shows this. 

 
{noformat}
[2019-07-17 17:09:54,644] {__init__.py:1374} INFO - Executing <Task(BashOperator): sample_task> on 2019-07-17T17:09:48.710579+00:00
[2019-07-17 17:09:54,644] {base_task_runner.py:119} INFO - Running: [u'sudo', u'-E', u'-H', u'-u', 'myuser', u'airflow', u'run', 'sample-team-sample-pipeline', 'sample_task', '2019-07-17T17:09:48.710579+00:00', u'--job_id', '471', u'--raw', u'-sd', u'DAGS_FOLDER/sample_team/sample-dag.py', u'--cfg_path', '/tmp/tmpZe8CVI']
[2019-07-17 17:09:54,662] {base_task_runner.py:101} INFO - Job 471: Subtask sample_task sudo: airflow: command not found {noformat}
Tasks which doesn't have the run_as_user setting runs alright. Workers where airflow is globally installed(and not in a virtual env) works alright which probably suggests me that the PATH variable is not being passed down when the worker triggers the actual task bit with sudo. 

 

Is there a way to get around this?



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)