You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ROHIT K RAMWAL (JIRA)" <ji...@apache.org> on 2019/06/11 18:35:00 UTC
[jira] [Comment Edited] (AIRFLOW-4772) airflow run_as_user creates
new config file when trying to execute a task
[ https://issues.apache.org/jira/browse/AIRFLOW-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16861310#comment-16861310 ]
ROHIT K RAMWAL edited comment on AIRFLOW-4772 at 6/11/19 6:34 PM:
------------------------------------------------------------------
[~ash] I tried to run it. Here is scenario where its failing.
python3 /bin/airflow run hive_test_flow hiveactiontobackuptable 2019-06-11T18:18:43.304403+00:00 --job_id 71 --raw -sd DAGS_FOLDER/hive_test/test_hive.py --cfg_path /tmp/tmpwz67adw6
The cfg is a showing for user id location inside the tmpwz67adw6 and not the default.
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 32, in <module>
args.func(args)
File "/usr/local/lib/python3.6/site-packages/airflow/utils/cli.py", line 74, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 501, in run
dag = get_dag(args)
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 145, in get_dag
'parse.'.format(args.dag_id))
airflow.exceptions.AirflowException: dag_id could not be found: hive_test_flow. Either the dag did not exist or it failed to parse.
was (Author: rramwal):
[~ash] I tried to run it. Here is scenario where its failing.
python3 /bin/airflow run hive_test_flow hiveactiontobackuptable 2019-06-11T18:18:43.304403+00:00 --job_id 71 --raw -sd DAGS_FOLDER/hive_test/test_hive.py --cfg_path /tmp/tmpwz67adw6
The cfg is a showing for user id location inside the tmpwz67adw6 and not the default.
> airflow run_as_user creates new config file when trying to execute a task
> -------------------------------------------------------------------------
>
> Key: AIRFLOW-4772
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4772
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: 1.10.3
> Environment: Linux centos-7.2.1511
> Reporter: ROHIT K RAMWAL
> Priority: Major
>
> I am using celery executor to scale airflow with redis as backed. Workers are executed with airflow user and have priviliges to sudo to other users in the system.
> In my config i have set the default_impersonation = 'svc*123' which is a default load id for all my jobs. When ever i trigger a job the airflow creates a new folder under 'svc*123' and generates a default config which fails to identify the original configuration.
> Tried the same with run_as_user which also creates new config first time. If we use multiple users to administrate our system better airflow keeps on generating new config and default database everytime.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)