You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/29 06:51:14 UTC
[GitHub] [airflow] shubhashish9 opened a new issue, #26769: AIrflow 2.4 crashes python while making requests.get call
shubhashish9 opened a new issue, #26769:
URL: https://github.com/apache/airflow/issues/26769
### Apache Airflow version
2.4.0
### What happened
In my DAG I make a request.get call and it fails with sigsev. In the background python crashes.
I tried with debug executor and that works, But in python virtualenv or in docker it fails with same error.
### What you think should happen instead
This sigsev should not have happened.
### How to reproduce
Using this dag script the issue is reproducible:
from airflow import DAG
from datetime import datetime
from airflow.operators.python_operator import PythonOperator
import requests
def test_request(**context):
url='https://jsonplaceholder.typicode.com/todos/1'
response = requests.get(
url=url
)
default_args = {'start_date': datetime(2022, 5, 11)}
dag = DAG(
dag_id='s3-test-24',
schedule_interval=None,
default_args=default_args,
start_date=datetime(2022, 5, 11)
)
write_file_func_task = PythonOperator(
task_id='test_request_task_24',
python_callable=test_request,
provide_context=True,
dag=dag
)
write_file_func_task
### Operating System
ProductName: macOS ProductVersion: 12.4 BuildVersion: 21F79
### Versions of Apache Airflow Providers
apache-airflow-providers-cncf-kubernetes==4.3.0
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-sqlite==3.2.
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else
It is reproducible consistently
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #26769: AIrflow 2.4 crashes python while making requests.get call
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #26769: AIrflow 2.4 crashes python while making requests.get call
URL: https://github.com/apache/airflow/issues/26769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26769: AIrflow 2.4 crashes python while making requests.get call
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26769:
URL: https://github.com/apache/airflow/issues/26769#issuecomment-1261890835
This is a problem with Python SSL on MacOS M1 not Airflow. We cannot do much about it because this is an environmental thing.
You can try various workaround and approaches that mitigate the issue - you can google it, here are two likely ways you can workaround it (read the discussions/various approaches there). But you can find plenty of other discussions/workarounds for similar issues people have (not only in Airflow - this problem affects everyone).
* https://stackoverflow.com/questions/69598150/apple-mac-book-pro-m1-chip-segmentation-error-when-using-python-requests-library
* https://github.com/apache/airflow/discussions/24463
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26769: AIrflow 2.4 crashes python while making requests.get call
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26769:
URL: https://github.com/apache/airflow/issues/26769#issuecomment-1261896126
FYI. Just to summarise the above. Airflow's Docker container already has a good openssl installed. So most likely just updating your openssl in MacOS and simply usig Python 3.9 or 3.10 should solve the problem. But refer to the solutions that others posted on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] shubhashish9 commented on issue #26769: AIrflow 2.4 crashes python while making requests.get call
Posted by GitBox <gi...@apache.org>.
shubhashish9 commented on issue #26769:
URL: https://github.com/apache/airflow/issues/26769#issuecomment-1261854037
Attaching scheduler debug logs and dag logs
[dag_id=s3-test-24_run_id=manual__2022-09-29T06_55_11.930070+00_00_task_id=test_request_task_24_attempt=1.log](https://github.com/apache/airflow/files/9671932/dag_id.s3-test-24_run_id.manual__2022-09-29T06_55_11.930070%2B00_00_task_id.tes
[schedulerLogs.log](https://github.com/apache/airflow/files/9671934/schedulerLogs.log)
t_request_task_24_attempt.1.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #26769: AIrflow 2.4 crashes python while making requests.get call
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #26769:
URL: https://github.com/apache/airflow/issues/26769#issuecomment-1261843818
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org