You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Maximilian Roos (JIRA)" <ji...@apache.org> on 2017/10/26 21:55:00 UTC
[jira] [Updated] (AIRFLOW-1758) Print full traceback on errors
[ https://issues.apache.org/jira/browse/AIRFLOW-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Maximilian Roos updated AIRFLOW-1758:
-------------------------------------
Description:
Currently when there is a failure during a run, it's difficult to see what the cause was. Could we at least print the python stack trace?
As an example:
{code:python}
[2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with return code 1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py", line 192, in execute
self.py_file, self.py_options)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 155, in start_python_dataflow
task_id, variables, dataflow, name, ["python"] + py_options)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 141, in _start_dataflow
_Dataflow(cmd).wait_for_done()
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 122, in wait_for_done
self._proc.returncode))
Exception: DataFlow failed with return code 1
{code}
I then need to jump into a repl and attempt simulate the command that airflow would have run, which is both difficult and error prone. (Or is there a simpler way of doing this??)
I then get a better stack-trace:
{code:python}
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line 328, in run
return self.runner.run(self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 283, in run
self.dataflow_client.create_job(self.job), self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py", line 168, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 423, in create_job
self.create_job_description(job)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 446, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 347, in stage_job_resources
build_setup_args)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 439, in _build_setup_package
os.chdir(os.path.dirname(setup_file))
OSError: [Errno 2] No such file or directory: ''
{code}
Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
(I'm using the DataFlowPythonOperator at the moment, but I suspect the issue is wider)
was:
Currently when there is a failure during a run, it's difficult to see what the cause was. Could we at least print the python stack trace?
As an example:
```
[2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with return code 1
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py", line 192, in execute
self.py_file, self.py_options)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 155, in start_python_dataflow
task_id, variables, dataflow, name, ["python"] + py_options)
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 141, in _start_dataflow
_Dataflow(cmd).wait_for_done()
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 122, in wait_for_done
self._proc.returncode))
Exception: DataFlow failed with return code 1
```
I then need to jump into a repl and attempt simulate the command that airflow would have run, which is both difficult and error prone. (Or is there a simpler way of doing this??)
I then get a better stack-trace:
```
File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line 328, in run
return self.runner.run(self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 283, in run
self.dataflow_client.create_job(self.job), self)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py", line 168, in wrapper
return fun(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 423, in create_job
self.create_job_description(job)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 446, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 347, in stage_job_resources
build_setup_args)
File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 439, in _build_setup_package
os.chdir(os.path.dirname(setup_file))
OSError: [Errno 2] No such file or directory: ''
```
Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
(I'm using the DataFlowPythonOperator at the moment, but I suspect the issue is wider)
> Print full traceback on errors
> ------------------------------
>
> Key: AIRFLOW-1758
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1758
> Project: Apache Airflow
> Issue Type: Improvement
> Components: cli
> Reporter: Maximilian Roos
> Priority: Minor
>
> Currently when there is a failure during a run, it's difficult to see what the cause was. Could we at least print the python stack trace?
> As an example:
> {code:python}
> [2017-10-26 21:43:38,155] {models.py:1563} ERROR - DataFlow failed with return code 1
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1461, in _run_raw_task
> result = task_copy.execute(context=context)
> File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/dataflow_operator.py", line 192, in execute
> self.py_file, self.py_options)
> File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 155, in start_python_dataflow
> task_id, variables, dataflow, name, ["python"] + py_options)
> File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 141, in _start_dataflow
> _Dataflow(cmd).wait_for_done()
> File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_dataflow_hook.py", line 122, in wait_for_done
> self._proc.returncode))
> Exception: DataFlow failed with return code 1
> {code}
> I then need to jump into a repl and attempt simulate the command that airflow would have run, which is both difficult and error prone. (Or is there a simpler way of doing this??)
> I then get a better stack-trace:
> {code:python}
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/pipeline.py", line 328, in run
> return self.runner.run(self)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 283, in run
> self.dataflow_client.create_job(self.job), self)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/utils/retry.py", line 168, in wrapper
> return fun(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 423, in create_job
> self.create_job_description(job)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/apiclient.py", line 446, in create_job_description
> job.options, file_copy=self._gcs_file_copy)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 347, in stage_job_resources
> build_setup_args)
> File "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/dataflow/internal/dependency.py", line 439, in _build_setup_package
> os.chdir(os.path.dirname(setup_file))
> OSError: [Errno 2] No such file or directory: ''
> {code}
> Somewhat related to: https://issues.apache.org/jira/browse/AIRFLOW-174
> (I'm using the DataFlowPythonOperator at the moment, but I suspect the issue is wider)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)