You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/13 14:36:35 UTC
[GitHub] [airflow] ranamit112 opened a new issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
ranamit112 opened a new issue #13045:
URL: https://github.com/apache/airflow/issues/13045
**Apache Airflow version**: 1.10.7
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N.A.
**Environment**:
- **Cloud provider or hardware configuration**: N.A.
- **OS** (e.g. from /etc/os-release): Red Hat Enterprise Linux Server 7.6 (Maipo)
- **Kernel** (e.g. `uname -a`): 3.10.0-957.35.2.el7.x86_64
- **Install tools**: pip, python3.7
- **Others**:
**What happened**: Could not install airflow 1.10.7 using constraint file for 1.10.7.
```
ERROR: Could not find a version that satisfies the requirement future<0.17,>=0.16.0 (from apache-airflow[celery,postgres,statsd])
ERROR: No matching distribution found for future<0.17,>=0.16.0
```
**What you expected to happen**: pip install success
<!-- What do you think went wrong? -->
The 1.10.7 [constraint file](https://github.com/apache/airflow/blob/constraints-1.10.7/constraints-3.7.txt) (that was created using 1.10.10 dependencies) includes a constraint for `future==0.18.2` while the [setup.py](https://github.com/apache/airflow/blob/1.10.7/setup.py#L467) for that version includes the requirement that `future>=0.16.0, <0.17`, which is incompatible.
This can be considered part of https://github.com/apache/airflow/issues/12838, as this happens only when using the new pip 20.3 resolver, but I think pip is behaving correctly, and the real problem is in the constraint file.
**How to reproduce it**:
```
python3.7 -m venv venv_python37_airflow
cd venv_python37_airflow
. ./bin/activate
python3.7 -m pip install --upgrade pip
python3.7 -m pip install wheel
AIRFLOW_VERSION=1.10.7
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
python -m pip install "apache-airflow[celery,postgres,statsd]==${AIRFLOW_VERSION}" --constraint ${CONSTRAINT_URL}
```
**Anything else we need to know**:
From my testing removing the constraint for `future==0.18.2` solved this meaning it's the only non-valid constraint for this version.
I believe this happens in other earlier versions, but I did not test version earlier than 1.10.7.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13045:
URL: https://github.com/apache/airflow/issues/13045
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744028733
Thanks for reporting this. I will leave it for reference to anone that will try to install Airflow 1.10.7, however, our recommendation is to move forward with 1.10.14 (bridge release for 2.0) with the goal of moving to 2.0 as soon as possible. This is the release where constraints have been finally solved once and for all (also in the 2.0 version) and we highly recommend migrating to 2.0. 2.0.0rc2 is being voted and it is very likely the official 2.0.0 of Airflow will be released Monday/Tuesday. 1.10.14 is already released last week.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744028733
Thanks for reporting this. I will leave it for reference to anone that will try to install Airflow 1.10.7, however our recommendation is to move forward with 1.10.14 (bridge release for 2.0) with the goal of moving to 2.0 a. This is the release where constraints have been finally solved once and for all (also in the 2.0 version) and we highly recommend migrating to 2.0. 2.0.0rc2 is being voted and it is very likely the official 2.0.0 of Airflow will be released Monday/Tuesday. 1.10.14 is already released last week.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744591339
> While I don't doubt that there are several problems with pip 20.3, as you documented in the attached issues, I don't think this one falls under them.
> The constraints file for 1.10.7 is plainly and simply wrong, which is not surprising since it was created for 1.10.10.
> As I wrote above it includes a constraint for `future==0.18.2` while the setup.py for that version includes the requirement for `future>=0.16.0, <0.17`, so any good resolver would indicate that there is a mismatch.
The problem is that before 20.3 PIP was happily accepting and resolving this anyway.
But I think there is nothing to fix now anyway. 1.10.7 is prehistory and we are recommending everyone to migrate to 1.10.14 which has good constraints (those constraints BTW do not work with pip 20.3 anyway). So similarly like with any other problems with previous releases - 1.10.14 + PIP 20.2.4 (for now) solves the problems we know about (including installation problems). I don't think any other action is needed at this stage.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744535315
> 1. I think this raises a good case to remove the tag contraints-1.10.7 (and earlier, probably), as the constraint files in them will not work when using pip 20.3, but it's your call.
Airflow installation does not work in general using PIP 20.3. This is a known issue, very prominently displayed on our site. https://github.com/apache/airflow/issues/12838 - if you tried pip 20.3, then that explains the problem.
We raised the issue (rather loudly) to the PIP team, They are going to release 20.3.2 today that fixes some of the problems we (and others) raised. I recommend to repeat the check once they release 20.3.2 or downgrade to 20.2.4 (or use legacy resolver) as this is officially the only version of PIP that Airflow works with.
> 2. Commenting a workaround for posterity, for anyone like me who needs to have a more verified version of 1.10.7 for CI or migration:
>
> ```
> curl ${CONSTRAINT_URL} > constraints.txt
> grep -v "future==" constraints.txt > constraints_good.txt # remove bad constraint
> python -m pip install "apache-airflow[celery,postgres,statsd]==${AIRFLOW_VERSION}" --constraint constraints_good.txt
> ```
There are many other packages which have problems with PIP 20.3. Until those problems are solved, I heartily recommend to downgrade PIP to 20.2.4 and keep it this way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744016704
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744535315
> 1. I think this raises a good case to remove the tag contraints-1.10.7 (and earlier, probably), as the constraint files in them will not work when using pip 20.3, but it's your call.
Airflow installation does not work in general using PIP 20.3. This is a known issue, very prominently displayed on our site. https://github.com/apache/airflow/issues/12838 - if you tried pip 20.3, then that explains the problem.
We raised the issue (rather loudly) to the PIP team, They are going to release 20.3.2 today that fixes some of the problems we (and others) raised. I recommend to repeat the check once they release 20.3.2 or downgrade to 20.2.4 (or use legacy resolver) as this is officially the only version of PIP that Airflow works with.
> 2. Commenting a workaround for posterity, for anyone like me who needs to have a more verified version of 1.10.7 for CI or migration:
>
> ```
> curl ${CONSTRAINT_URL} > constraints.txt
> grep -v "future==" constraints.txt > constraints_good.txt # remove bad constraint
> python -m pip install "apache-airflow[celery,postgres,statsd]==${AIRFLOW_VERSION}" --constraint constraints_good.txt
> ```
There are many other packages which have problems with PIP 20.3. Until those problems are solved. I heartily recommend to downgrade PIP to 20.2.4 and keep it this way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ranamit112 edited a comment on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
ranamit112 edited a comment on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744552467
While I don't doubt that there are several problems with pip 20.3, as you documented in the attached issues, I don't think this one falls under them.
The constraints file for 1.10.7 is plainly and simply wrong, which is not surprising since it was created for 1.10.10.
As I wrote above it includes a constraint for `future==0.18.2` while the setup.py for that version includes the requirement for `future>=0.16.0, <0.17`, so any good resolver would indicate that there is a mismatch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ranamit112 commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
ranamit112 commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744522885
1. I think this raises a good case to remove the tag contraints-1.10.7 (and earlier, probably), as the constraint files in them will not work when using pip 20.3, but it's your call.
2. Commenting a workaround for posterity, for anyone like me who needs to have a more verified version of 1.10.7 for CI or migration:
```
curl ${CONSTRAINT_URL} > constraints.txt
grep -v "future==" constraints.txt > constraints_good.txt # remove bad constraint
python -m pip install "apache-airflow[celery,postgres,statsd]==${AIRFLOW_VERSION}" --constraint constraints_good.txt
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ranamit112 commented on issue #13045: Airflow constraints file constraints-1.10.7/constraints-3.7 incorrect constraints preventing install
Posted by GitBox <gi...@apache.org>.
ranamit112 commented on issue #13045:
URL: https://github.com/apache/airflow/issues/13045#issuecomment-744552467
While I don't doubt that there are several problems with pip 20.3, as you documented in the attached issues, I don't think this one falls under then.
The constraints file for 1.10.7 is plainly and simply wrong, which is not surprising since it was created for 1.10.10.
As I wrote above it includes a constraint for `future==0.18.2` while the setup.py for that version includes the requirement for `future>=0.16.0, <0.17`, so any good resolver would indicate that there is a mismatch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org