You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/14 13:08:55 UTC
[GitHub] [airflow] ee07dazn commented on issue #12761: Cloudwatch Logs Integration Does Not Write Logs With CeleryExecutor (2.0.0)
ee07dazn commented on issue #12761:
URL: https://github.com/apache/airflow/issues/12761#issuecomment-744428254
@MrManicotti : Thanks for raising this ticket. I have a similar problem, whereby even in Sequential/Local executor, only INFO is getting written to Cloudwatch. Any "ERROR" messages are not getting written with remote logging as shown below
```
Dependencies all met for <TaskInstance: tutorial.test_snow 2020-12-13T07:00:00+00:00 [queued]>
Dependencies all met for <TaskInstance: tutorial.test_snow 2020-12-13T07:00:00+00:00 [queued]>
--------------------------------------------------------------------------------
Starting attempt 1 of 1
--------------------------------------------------------------------------------
Executing <Task(VYGSnowflakeOperator): test_snow> on 2020-12-13T07:00:00+00:00
Started process 161 to run task
Task exited with return code 1
```
Without remote logging, i get the following output:
```
*** Reading local file: /usr/local/airflow/logs/tutorial/print_date/2020-12-13T07:00:00+00:00/1.log
[2020-12-14 13:07:33,907] {{taskinstance.py:670}} INFO - Dependencies all met for <TaskInstance: tutorial.print_date 2020-12-13T07:00:00+00:00 [queued]>
[2020-12-14 13:07:33,937] {{taskinstance.py:670}} INFO - Dependencies all met for <TaskInstance: tutorial.print_date 2020-12-13T07:00:00+00:00 [queued]>
[2020-12-14 13:07:33,937] {{taskinstance.py:880}} INFO -
--------------------------------------------------------------------------------
[2020-12-14 13:07:33,937] {{taskinstance.py:881}} INFO - Starting attempt 1 of 1
[2020-12-14 13:07:33,937] {{taskinstance.py:882}} INFO -
--------------------------------------------------------------------------------
[2020-12-14 13:07:33,958] {{taskinstance.py:901}} INFO - Executing <Task(BashOperator): print_date> on 2020-12-13T07:00:00+00:00
[2020-12-14 13:07:33,963] {{standard_task_runner.py:54}} INFO - Started process 373 to run task
[2020-12-14 13:07:34,100] {{standard_task_runner.py:77}} INFO - Running: ['airflow', 'run', 'tutorial', 'print_date', '2020-12-13T07:00:00+00:00', '--job_id', '2', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/tuto.py', '--cfg_path', '/tmp/tmpvcretdrv']
[2020-12-14 13:07:34,101] {{standard_task_runner.py:78}} INFO - Job 2: Subtask print_date
[2020-12-14 13:07:34,220] {{logging_mixin.py:112}} INFO - Running %s on host %s <TaskInstance: tutorial.print_date 2020-12-13T07:00:00+00:00 [running]> 4624a16308a0
[2020-12-14 13:07:34,267] {{bash_operator.py:113}} INFO - Tmp dir root location:
/tmp
[2020-12-14 13:07:34,268] {{bash_operator.py:136}} INFO - Temporary script location: /tmp/airflowtmph5p8i1j5/print_datejbg3bgyq
[2020-12-14 13:07:34,268] {{bash_operator.py:146}} INFO - Running command: date2
[2020-12-14 13:07:34,282] {{bash_operator.py:153}} INFO - Output:
[2020-12-14 13:07:34,286] {{bash_operator.py:157}} INFO - /tmp/airflowtmph5p8i1j5/print_datejbg3bgyq: line 1: date2: command not found
[2020-12-14 13:07:34,287] {{bash_operator.py:161}} INFO - Command exited with return code 127
**[2020-12-14 13:07:34,303] {{taskinstance.py:1150}} ERROR - Bash command failed
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.7/site-packages/airflow/operators/bash_operator.py", line 165, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed**
[2020-12-14 13:07:34,306] {{taskinstance.py:1194}} INFO - Marking task as FAILED. dag_id=tutorial, task_id=print_date, execution_date=20201213T070000, start_date=20201214T130733, end_date=20201214T130734
[2020-12-14 13:07:38,807] {{local_task_job.py:102}} INFO - Task exited with return code 1
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org