You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2019/06/21 17:11:38 UTC
[GitHub] [airflow] jzucker2 edited a comment on issue #5452: [AIRFLOW-4829]
More descriptive exceptions for EMR sensors
jzucker2 edited a comment on issue #5452: [AIRFLOW-4829] More descriptive exceptions for EMR sensors
URL: https://github.com/apache/airflow/pull/5452#issuecomment-504488976
@ashb something like this:
```
[2019-06-21 17:02:56,781] {emr_base_sensor.py:33} INFO - Job flow currently TERMINATED_WITH_ERRORS
[2019-06-21 17:02:56,799] {__init__.py:1580} ERROR - EMR job failed for code: BOOTSTRAP_FAILURE with message Master instance (i-02c25cade3cbfc453) failed attempting to download bootstrap action 1 file from S3
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python2.7/dist-packages/airflow/sensors/base_sensor_operator.py", line 108, in execute
while not self.poke(context):
File "/usr/local/lib/python2.7/dist-packages/airflow/sensors/emr_base_sensor.py", line 43, in poke
raise AirflowException(final_message)
AirflowException: EMR job failed for code: BOOTSTRAP_FAILURE with message Master instance (i-02c25cade3cbfc453) failed attempting to download bootstrap action 1 file from S3
[2019-06-21 17:02:56,801] {__init__.py:1603} INFO - Marking task as UP_FOR_RETRY
[2019-06-21 17:02:57,012] {logging_mixin.py:95} INFO - [2019-06-21 17:02:57,011] {sendgrid.py:107} INFO - Email with subject Airflow alert: <TaskInstance: hourly_blocks_setup_integration_v1.job_flow_sensor 2019-06-21T16:52:17.574713+00:00 [up_for_retry]> is successfully sent to recipients: [{'to': [{'email': u'test@test.com'}, {'email': u'test2@test.com'}]}]
[2019-06-21 17:02:57,030] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor Traceback (most recent call last):
[2019-06-21 17:02:57,031] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/bin/airflow", line 32, in <module>
[2019-06-21 17:02:57,031] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor args.func(args)
[2019-06-21 17:02:57,031] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/utils/cli.py", line 74, in wrapper
[2019-06-21 17:02:57,031] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor return f(*args, **kwargs)
[2019-06-21 17:02:57,031] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 523, in run
[2019-06-21 17:02:57,032] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor _run(args, dag, ti)
[2019-06-21 17:02:57,032] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 442, in _run
[2019-06-21 17:02:57,032] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor pool=args.pool,
[2019-06-21 17:02:57,033] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 73, in wrapper
[2019-06-21 17:02:57,033] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor return func(*args, **kwargs)
[2019-06-21 17:02:57,033] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/models/__init__.py", line 1441, in _run_raw_task
[2019-06-21 17:02:57,034] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor result = task_copy.execute(context=context)
[2019-06-21 17:02:57,034] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/sensors/base_sensor_operator.py", line 108, in execute
[2019-06-21 17:02:57,034] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor while not self.poke(context):
[2019-06-21 17:02:57,034] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor File "/usr/local/lib/python2.7/dist-packages/airflow/sensors/emr_base_sensor.py", line 43, in poke
[2019-06-21 17:02:57,035] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor raise AirflowException(final_message)
[2019-06-21 17:02:57,035] {base_task_runner.py:101} INFO - Job 67389: Subtask job_flow_sensor airflow.exceptions.AirflowException: EMR job failed for code: BOOTSTRAP_FAILURE with message Master instance (i-02c25cade3cbfc453) failed attempting to download bootstrap action 1 file from S3
[2019-06-21 17:02:58,890] {logging_mixin.py:95} INFO - [2019-06-21 17:02:58,890] {jobs.py:2562} INFO - Task exited with return code 1
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services