[GitHub] [airflow] kshirsagarmahesh commented on issue #10976: Airflow bash operator to log error when command exits with non zero exit code

[GitBox <gi...@apache.org>]

kshirsagarmahesh commented on issue #10976:
URL: https://github.com/apache/airflow/issues/10976#issuecomment-694448079


   Thanks @kaxil - This works for most situations. However I found a unique situation where this does not log a error with standard message saying the dag is FAILED. 
   Scenario: Bash Operator with run_as_user option. The user I am using does not have sudo permissions. In such case the log remains incomplete:
   Here is complete log:
   ------------------------------------------------------------------------------
   [2020-09-17 15:11:37,196] {taskinstance.py:901} INFO - Executing <Task(BashOperator): sudo_fail_test> on 2020-09-17T19:11:23.017017+00:00
   [2020-09-17 15:11:37,196] {base_task_runner.py:131} INFO - Running on host: devmachine.company.com
   [2020-09-17 15:11:37,196] {base_task_runner.py:132} INFO - Running: ['sudo', '-E', '-H', '-u', 'newuser', 'PYTHONPATH=/xx/rda/airflow/airflow_env/lib/python3.7/site-packages/:/opt/xx/lib/python3.7/site-packages/', 'airflow', 'run', 'ca_exit_code_test', 'sudo_fail_test', '2020-09-17T19:11:23.017017+00:00', '--job_id', '8332', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/ca_exit_code_test.py', '--cfg_path', '/tmp/tmpputqd5lz']
   [2020-09-17 15:11:37,288] {base_task_runner.py:113} INFO - Job 8332: Subtask sudo_fail_test testuser is not in the sudoers file.  This incident will be reported.
   [2020-09-17 15:11:42,205] {local_task_job.py:102} INFO - Task exited with return code 1
   
   ------------------------------------------------------------------------------
   
   The dag I used had this task:
   with models.DAG(
           dag_id='ca_exit_code_test',
           default_args=default_args,
           schedule_interval='0 10 5 * *',
           concurrency=3,
           max_active_runs=1,
           catchup=False
   ) as dag:
       start = DummyOperator(
           task_id='start'
       )
       task_1 = BashOperator(
               task_id='sudo_fail_test' ,
               bash_command = 'ls' ,
               dag = dag,
               run_as_user= 'newuser'
                )
       start >> task_1 
   As you can see the log is incomplete. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org