You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/12 12:25:48 UTC
[GitHub] [airflow] kaxil opened a new pull request #17575: Make `pandas` an optional core dependency
kaxil opened a new pull request #17575:
URL: https://github.com/apache/airflow/pull/17575
We only use `pandas` in `DbApiHook.get_pandas_df`. Not all users use it, plus
while `pandas` now supports many pre-compiled packages it still can take forever where
it needs to be compiled.
So for first-time users this can be a turn off. If pandas is already installed this
will work fine, but if not users have an option to run `pip install apache-airflow[pandas]`
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922540200
> > We install that in Dockerfile so we have it while running tests
>
> Agree. The "test" environment has all the deps in - including pandas and numpy. The "test" dependency ([devel_all] extra) defines all things needed to run tests.
>
> I think while numpy is not needed as "airflow" dependency, it's perfectly fine to have it as "test airflow" dependency.
@kaxil @potiuk Thanks for the feedback. The reason I ask is because of the discussion here: https://github.com/apache/airflow/pull/18343#discussion_r711648377. For the test environment there is no issue as you've mentioned, so I guess the concern there is about the local development environment? Of course, if you run the test locally and numpy is missing, you can just install it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922529365
> @kaxil Should this import also be put inside a try block, if `numpy` is not a dependency required in the `devel` extra?
>
> https://github.com/apache/airflow/blob/11e34535e8cda2f22b26eb3f951a952e3acfe333/tests/utils/test_json.py#L24
We install that in Dockerfile so we have it while running tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748
> We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps installed in the docker image (which is 1-1 replica of CI test environment), and https://github.com/apache/airflow/blob/main/LOCAL_VIRTUALENV.rst#creating-a-local-virtualenv where we explain that you need to add "devel" extra if you want to install local virtualenv for development.
>
> So I think all is good here
@potiuk Thanks. One clarification - from https://github.com/apache/airflow/pull/18343#discussion_r711648377, I got the impression that `devel` extra contains the libraries needed for core testing (in local virtualenv). Does `test_json.py` count as core testing? I think that's where I'm confused about why numpy only needs to be installed in `devel_all`, but not `devel`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897785664
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #17575:
URL: https://github.com/apache/airflow/pull/17575
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922550696
> Does test_json.py count as core testing?
Depends what you mean by "core". There is no mentioning of "core" in the extras doc:
From https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html#bundle-extras
extra | install command | enables
-- | -- | --
all | pip install 'apache-airflow[all]' | All Airflow user facing features (no devel and doc requirements)
all_dbs | pip install 'apache-airflow[all_dbs]' | All databases integrations
devel | pip install 'apache-airflow[devel]' | Minimum dev tools requirements (without providers)
devel_hadoop | pip install 'apache-airflow[devel_hadoop]' | Same as devel + dependencies for developing the Hadoop stack
devel_all | pip install 'apache-airflow[devel_all]' | Everything needed for development (devel_hadoop + providers)
devel_ci | pip install 'apache-airflow[devel_ci]' | All dependencies required for CI build.
It's clear that `devel` is for all tests except providers. the `test.json` is not in providers. Do you think it's ambiguous? maybe you would like to improve the docs if things are not clear enough ? We have more than 1700 contibutors and clarifications from users like you who have doubts, are most welcome. Do you think this can be improved in the docs? How?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#discussion_r687978415
##########
File path: setup.py
##########
@@ -395,6 +395,11 @@ def write_version(filename: str = os.path.join(*[my_dir, "airflow", "git_version
pagerduty = [
'pdpyras>=4.1.2,<5',
]
+pandas = [
+ # Pandas stopped releasing 3.6 binaries for 1.2.* series.
+ 'pandas>=0.17.1, <1.2;python_version<"3.7"',
+ 'pandas>=0.17.1, <2.0;python_version>="3.7"',
Review comment:
```suggestion
'pandas>=0.17.1, <2.0',
```
Pandas 1.2+ provides the `Requires-Python` metadata, so 1.2+ versions will be automatically excluded when `pip install` on Python 3.6:
```
$ python3.6 -m venv x
$ . x/bin/activate
(x) $ pip install pandas
Collecting pandas
Using cached https://.../pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl
Collecting python-dateutil>=2.7.3 (from pandas)
...
```
So the `python_version<"3.7"` line is not needed.
Note: Python 3.6.0 ships with pip 9.1 by default, which supports `Requires-Python`, so it’s highly unlikely someone on 3.6 running on pip version not supporting this metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897798876
I _think_ this closes #12500 - are there any other uses of numpy in the code base?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748
> We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps installed in the docker image (which is 1-1 replica of CI test environment), and https://github.com/apache/airflow/blob/main/LOCAL_VIRTUALENV.rst#creating-a-local-virtualenv where we explain that you need to add "devel" extra if you want to install local virtualenv for development.
>
> So I think all is good here
@potiuk Thanks. One clarification - from https://github.com/apache/airflow/pull/18343#discussion_r711648377, I got the impression that `devel` extra contains the libraries needed for core Airflow testing (in local virtualenv). Does `test_json.py` count as core Airflow testing? I think that's where I'm confused about why numpy isn't needed in `devel` extra. (I understand it's part of `devel_all`, so no issues there.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748
> We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps installed in the docker image (which is 1-1 replica of CI test environment), and https://github.com/apache/airflow/blob/main/LOCAL_VIRTUALENV.rst#creating-a-local-virtualenv where we explain that you need to add "devel" extra if you want to install local virtualenv for development.
>
> So I think all is good here
@potiuk Thanks. One clarification - from https://github.com/apache/airflow/pull/18343#discussion_r711648377, I got the impression that `devel` extra contains the libraries needed for core Airflow testing (in local virtualenv). Does `test_json.py` count as core Airflow testing? I think that's where I'm confused about why numpy is only needed in `devel_all`, but not `devel`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922516186
@kaxil Should this import also be put inside a try block, since `numpy` is not a dependency required in the `devel` extra? https://github.com/apache/airflow/blob/11e34535e8cda2f22b26eb3f951a952e3acfe333/tests/utils/test_json.py#L24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922542839
> I guess the concern there is about the local development environment? Of course, if you run a test locally and numpy is missing, you can just install it.
We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps installed in the docker image (which is 1-1 replica of CI test environment), and https://github.com/apache/airflow/blob/main/LOCAL_VIRTUALENV.rst#creating-a-local-virtualenv where we explain that you need to add "devel" extra if you want to install local virtualenv for development.
So I think all is good here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897800623
> I _think_ this closes #12500 - are there any other uses of numpy in the code base?
Not yet. It is used at least in AirflowJsonEncoder 😱
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897788822
And it should work for old airflow versions as well. AFAIK when you specifiy extra that is missing, it will be silently ignored, so it will work for pre 2.2 Airflow versions as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922516186
@kaxil Should this import also be put inside a try block, since `numpy` is not a dependency needed in the `devel` extra? https://github.com/apache/airflow/blob/11e34535e8cda2f22b26eb3f951a952e3acfe333/tests/utils/test_json.py#L24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922555221
> Depends what you mean by "core". There is no mentioning of "core" in the extras doc:
Ah okay, that is a good point.
> It's clear that `devel` is for all tests except providers. the `test.json` is not in providers. Do you think it's ambiguous? maybe you would like to improve the docs if things are not clear enough ? We have more than 1700 contibutors and clarifications from users like you who have doubts, are most welcome. Do you think this can be improved in the docs? How?
Sounds good. If I come up with something concrete, I'll submit a PR.
Thanks for clarifying my questions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897945427
https://github.com/apache/airflow/pull/17575/commits/398ebfe0a7e36b8195801d4a3b79cf335facaae1 should make numpy optional as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#discussion_r687978415
##########
File path: setup.py
##########
@@ -395,6 +395,11 @@ def write_version(filename: str = os.path.join(*[my_dir, "airflow", "git_version
pagerduty = [
'pdpyras>=4.1.2,<5',
]
+pandas = [
+ # Pandas stopped releasing 3.6 binaries for 1.2.* series.
+ 'pandas>=0.17.1, <1.2;python_version<"3.7"',
+ 'pandas>=0.17.1, <2.0;python_version>="3.7"',
Review comment:
```suggestion
'pandas>=0.17.1, <2.0',
```
Pandas 1.2+ provides the `Requires-Python` metadata, so 1.2+ versions will be automatically excluded when `pip install` on Python 3.6:
```
$ python3.6 -m venv x
$ . x/bin/activate
(x) $ pip install pandas
Collecting pandas
Using cached https://.../pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl
Collecting python-dateutil>=2.7.3 (from pandas)
...
```
So the `python_version<"3.7"` line is not needed. (Although it should not have a negative impact either.)
Note: Python 3.6.0 ships with pip 9.1 by default, which supports `Requires-Python`, so it’s highly unlikely someone on 3.6 running on pip version not supporting this metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#discussion_r688053575
##########
File path: setup.py
##########
@@ -395,6 +395,11 @@ def write_version(filename: str = os.path.join(*[my_dir, "airflow", "git_version
pagerduty = [
'pdpyras>=4.1.2,<5',
]
+pandas = [
+ # Pandas stopped releasing 3.6 binaries for 1.2.* series.
+ 'pandas>=0.17.1, <1.2;python_version<"3.7"',
+ 'pandas>=0.17.1, <2.0;python_version>="3.7"',
Review comment:
Good point, updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-897788822
And the Dockerfile will continue to work for old airflow versions as well. AFAIK when you specifiy extra that is missing, it will be silently ignored, so it will work for pre 2.2 Airflow versions as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922538744
> We install that in Dockerfile so we have it while running tests
Agree. The "test" environment has all the deps in - including pandas and numpy. The "test" dependency ([devel_all] extra) defines all things needed to run tests.
I think while numpy is not needed as "airflow" dependency, it's perfectly fine to have it as "test airflow" dependency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922540200
> > We install that in Dockerfile so we have it while running tests
>
> Agree. The "test" environment has all the deps in - including pandas and numpy. The "test" dependency ([devel_all] extra) defines all things needed to run tests.
>
> I think while numpy is not needed as "airflow" dependency, it's perfectly fine to have it as "test airflow" dependency.
@kaxil @potiuk Thanks for the feedback. The reason I ask is because of the discussion here: https://github.com/apache/airflow/pull/18343#discussion_r711648377. For the test environment there is no issue as you've mentioned, so I guess the concern there is about the local development environment? Of course, if you run a test locally and numpy is missing, you can just install it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922545748
> We have PLENTY of tests that will fail when some extra is not installed. That's why we have https://github.com/apache/airflow/blob/main/BREEZE.rst with all deps installed in the docker image (which is 1-1 replica of CI test environment), and https://github.com/apache/airflow/blob/main/LOCAL_VIRTUALENV.rst#creating-a-local-virtualenv where we explain that you need to add "devel" extra if you want to install local virtualenv for development.
>
> So I think all is good here
@potiuk Thanks. One clarification - from https://github.com/apache/airflow/pull/18343#discussion_r711648377, I got the impression that `devel` extra contains the libraries needed for core Airflow testing (in local virtualenv). Does `test_json.py` count as core Airflow testing? I think that's where I'm confused about why numpy only needs to be installed in `devel_all`, but not `devel`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] edwardwang888 edited a comment on pull request #17575: Make `pandas` an optional core dependency
Posted by GitBox <gi...@apache.org>.
edwardwang888 edited a comment on pull request #17575:
URL: https://github.com/apache/airflow/pull/17575#issuecomment-922516186
@kaxil Should this import also be put inside a try block, if `numpy` is not a dependency required in the `devel` extra? https://github.com/apache/airflow/blob/11e34535e8cda2f22b26eb3f951a952e3acfe333/tests/utils/test_json.py#L24
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org