You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/12/08 12:43:31 UTC
[GitHub] [airflow] rrpelgrim opened a new issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
rrpelgrim opened a new issue #20134:
URL: https://github.com/apache/airflow/issues/20134
### Apache Airflow version
2.2.2 (latest released)
### What happened
My airflow tasks which creates matplotlib figures and saves them locally fails with the following error in the logs:
`INFO - Task exited with return code Negsignal.SIGABRT`
full traceback:
```
*** Reading local file: /Users/rpelgrim/airflow/logs/airflow_on_coiled_testing/visualize/2021-12-08T12:31:14.863577+00:00/1.log
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: airflow_on_coiled_testing.visualize manual__2021-12-08T12:31:14.863577+00:00 [queued]>
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1035} INFO - Dependencies all met for <TaskInstance: airflow_on_coiled_testing.visualize manual__2021-12-08T12:31:14.863577+00:00 [queued]>
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1241} INFO -
--------------------------------------------------------------------------------
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1242} INFO - Starting attempt 1 of 4
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1243} INFO -
--------------------------------------------------------------------------------
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1262} INFO - Executing <Task(_PythonDecoratedOperator): visualize> on 2021-12-08 12:31:14.863577+00:00
[2021-12-08, 13:35:23 UTC] {standard_task_runner.py:52} INFO - Started process 33014 to run task
[2021-12-08, 13:35:23 UTC] {standard_task_runner.py:76} INFO - Running: ['airflow', 'tasks', 'run', 'airflow_on_coiled_testing', 'visualize', 'manual__2021-12-08T12:31:14.863577+00:00', '--job-id', '77', '--raw', '--subdir', 'DAGS_FOLDER/airflow-coiled-testing.py', '--cfg-path', '/var/folders/ky/bqjn_gxn1xv0cn_8q5xvp3q40000gn/T/tmpzzle0cas', '--error-file', '/var/folders/ky/bqjn_gxn1xv0cn_8q5xvp3q40000gn/T/tmp9kwl1ww2']
[2021-12-08, 13:35:23 UTC] {standard_task_runner.py:77} INFO - Job 77: Subtask visualize
[2021-12-08, 13:35:23 UTC] {logging_mixin.py:109} INFO - Running <TaskInstance: airflow_on_coiled_testing.visualize manual__2021-12-08T12:31:14.863577+00:00 [running]> on host 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1427} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_EMAIL=richard@coiled.io
AIRFLOW_CTX_DAG_OWNER=rrpelgrim
AIRFLOW_CTX_DAG_ID=airflow_on_coiled_testing
AIRFLOW_CTX_TASK_ID=visualize
AIRFLOW_CTX_EXECUTION_DATE=2021-12-08T12:31:14.863577+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-12-08T12:31:14.863577+00:00
[2021-12-08, 13:35:23 UTC] {local_task_job.py:154} INFO - Task exited with return code Negsignal.SIGABRT
[2021-12-08, 13:35:23 UTC] {taskinstance.py:1270} INFO - Marking task as UP_FOR_RETRY. dag_id=airflow_on_coiled_testing, task_id=visualize, execution_date=20211208T123114, start_date=20211208T123523, end_date=20211208T123523
[2021-12-08, 13:35:23 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
### What you expected to happen
For the task to complete (since it runs fine locally in a jupyter notebook running on the same software env) and for the figure to be plotted and stored locally.
### How to reproduce
Reproduce with this DAG:
```python
from datetime import datetime, timedelta
import matplotlib.pyplot as plt
import numpy as np
from airflow.decorators import dag, task
# set default arguments to all tasks
default_args = {
'owner': 'rrpelgrim',
'depends_on_past': False,
'email_on_failure': False,
'email_on_retry': False,
'retries': 3,
'retry_delay': timedelta(minutes=5),
}
@dag(
default_args=default_args,
schedule_interval=None,
start_date=datetime(2021, 1, 1),
catchup=False,
)
def dag_name():
@task()
def visualize():
"""
Create visualisation plots and save as PNGs.
"""
# Create boxplot
data = np.arange(1,20)
fig, ax = plt.subplots(figsize=(10,10))
plt.title('Title', fontsize=15)
plt.boxplot(x=data)
plt.savefig('boxplot_test.png')
plt.close(fig)
print('Visualisations have been created.')
# Call task functions in order
viz = visualize()
# Call taskflow
demo = airflow_on_coiled_testing()
```
### Operating System
macOS 11.3.1
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
working locally with `airflow standalone`
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #20134:
URL: https://github.com/apache/airflow/issues/20134#issuecomment-988778850
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20134:
URL: https://github.com/apache/airflow/issues/20134#issuecomment-989357811
There is not much information but from your description it looks like this (or similar): https://stackoverflow.com/questions/60574601/how-can-i-fix-a-sigabrt-error-on-my-macos-hight-sierra
Simply Airflow tasks run as non-interactive process and they have no acces to your graphical card (and your UI). The opencv-headless package allows you to build the visualisation using only CPU.
You could also use your GPU likely but that requires some more complex setting and running an in-memory X Server with GPU acceleration (but you 'd need to dig much deeper to get it working).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rrpelgrim commented on issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
Posted by GitBox <gi...@apache.org>.
rrpelgrim commented on issue #20134:
URL: https://github.com/apache/airflow/issues/20134#issuecomment-989713211
Thanks @potiuk. Installing the `opencv-python-headless` package did not solve it for me, unfortunately.
I'm hesitant to dig into running it on my GPU. Do you jappen to know of another simpler workaround?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20134:
URL: https://github.com/apache/airflow/issues/20134#issuecomment-989863928
You should really check what's the error message there. You can try to run your stuff for example from a cron job to avoid the "interactive" part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20134: Negsignal.SIGABRT when trying to create matplotlib plots in DAG
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20134:
URL: https://github.com/apache/airflow/issues/20134#issuecomment-989864190
And I am moving that to discussion - this is not really an Airlfow bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org