You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/20 11:32:29 UTC
[GitHub] [airflow] ashb opened a new issue #12500: Investigate removing pandas dependency from Core Airflow
ashb opened a new issue #12500:
URL: https://github.com/apache/airflow/issues/12500
I want to take a look at removing numpy/pyarrow from _core_ airflow. We don't really need it, and it would make installing easier.
After 2.0
_Originally posted by @ashb in https://github.com/apache/airflow/issues/11950#issuecomment-731115568_
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek edited a comment on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
turbaszek edited a comment on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731641985
> And no pandas means no numpy right?
Yes, removing pandas will remove also numpy 👌
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jchacks edited a comment on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
jchacks edited a comment on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-775848779
https://github.com/apache/airflow/blob/7d38645472b0502212504b09d85b0e1271d74274/airflow/utils/json.py#L21
That line in utils imports numpy, not sure if this is relevant.
It was causing my webserver to crash since _my_ airflow version did not install numpy by default, for some reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #12500:
URL: https://github.com/apache/airflow/issues/12500
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731566726
Oh yeah! That will help with #11950 as well!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731641985
> And no pandas means no numpy right?
Probably. I forgot check it but no idea where we would need it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731578810
Searching for `pandas` import it seems that only `DbApiHook` requires it explicitly for one method: `get_pandas_df`.
Apart from that:
```
root@1572924b0714:/opt/airflow# pipdeptree -r -p pandas
pandas==1.1.0
- apache-airflow==2.0.0b3 [requires: pandas>=0.17.1,<2.0]
- nteract-scrapbook==0.4.1 [requires: pandas]
- pandas-gbq==0.13.2 [requires: pandas>=0.19.0]
```
Where `nteract-scrapbook` is required for `papermill` provider and `pandas-gbq` is for `google`. I removed pandas from setup.cfg and using clean venv installed airflow. No pandas was installed and running few airflow commands showed no errors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731641124
It probably used to be used more - in the old chart view I think, but we've removed those uses now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731641189
And no pandas means no numpy right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-731130902
+1 for that, I faced some dependency issues with pyarrow too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jchacks commented on issue #12500: Investigate removing pandas dependency from Core Airflow
Posted by GitBox <gi...@apache.org>.
jchacks commented on issue #12500:
URL: https://github.com/apache/airflow/issues/12500#issuecomment-775848779
https://github.com/apache/airflow/blob/7d38645472b0502212504b09d85b0e1271d74274/airflow/utils/json.py#L21
That line in utils imports numpy, not sure if this is relevant.
It was causing my webserver to crash since my airflow version doesnt install numpy by default.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org