You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/30 09:48:30 UTC
[GitHub] [airflow] tdbabas opened a new issue #11965: Airflow fails to initdb with cattrs 1.1.0
tdbabas opened a new issue #11965:
URL: https://github.com/apache/airflow/issues/11965
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
<!--
IMPORTANT!!!
PLEASE CHECK "SIMILAR TO X EXISTING ISSUES" OPTION IF VISIBLE
NEXT TO "SUBMIT NEW ISSUE" BUTTON!!!
PLEASE CHECK IF THIS ISSUE HAS BEEN REPORTED PREVIOUSLY USING SEARCH!!!
Please complete the next sections or the issue will be closed.
These questions are the first thing we need to know to understand the context.
-->
**Apache Airflow version**:
1.10.12
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
N/A
**Environment**:
- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release): Centos 7
- **Kernel** (e.g. `uname -a`): 3.10.0-229.el7.x86_64
- **Install tools**: pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
- **Others**:
**What happened**:
Following the instructions [here](http://airflow.apache.org/docs/stable/start.html), I encountered an issue at the `airflow initdb` stage:
```
[xxx@xxx ~]# airflow initdb
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 26, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python3.6/site-packages/airflow/bin/cli.py", line 94, in <module>
api_module = import_module(conf.get('cli', 'api_client')) # type: Any
File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/usr/local/lib/python3.6/site-packages/airflow/api/client/local_client.py", line 24, in <module>
from airflow.api.common.experimental import delete_dag
File "/usr/local/lib/python3.6/site-packages/airflow/api/common/experimental/delete_dag.py", line 26, in <module>
from airflow.models.serialized_dag import SerializedDagModel
File "/usr/local/lib/python3.6/site-packages/airflow/models/serialized_dag.py", line 35, in <module>
from airflow.serialization.serialized_objects import SerializedDAG
File "/usr/local/lib/python3.6/site-packages/airflow/serialization/serialized_objects.py", line 28, in <module>
import cattr
File "/usr/local/lib/python3.6/site-packages/cattr/__init__.py", line 1, in <module>
from .converters import Converter, GenConverter, UnstructureStrategy
File "/usr/local/lib/python3.6/site-packages/cattr/converters.py", line 16, in <module>
from attr import fields, resolve_types
ImportError: cannot import name 'resolve_types'
```
<!-- (please include exact error messages if you can) -->
**What you expected to happen**:
I expected the Airflow DB to be initialised as per the instructions.
<!-- What do you think went wrong? -->
**How to reproduce it**:
<!---
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If you are using kubernetes, please attempt to recreate the issue using minikube or kind.
## Install minikube/kind
- Minikube https://minikube.sigs.k8s.io/docs/start/
- Kind https://kind.sigs.k8s.io/docs/user/quick-start/
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
You can include images using the .md style of
To record a screencast, mac users can use QuickTime and then create an unlisted youtube video with the resulting .mov file.
--->
On a fresh installation, follow the Quick Start guide until the `apache initdb` stage.
**Anything else we need to know**:
Investigation suggests that this was caused by a Python dependency; specifically `cattrs==1.1.0`, which was released yesterday (2020-10-29). Downgrading `cattrs` manually to 1.0.0 does fix the issue and allows the Airflow database to be initialised:
```
[xxx@xxx ~]# pip3 install cattrs==1.0.0
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting cattrs==1.0.0
Downloading https://files.pythonhosted.org/packages/17/5b/6afbdaeb066ecf8ca28d85851048103ac80bb169491a54a14bd39823c422/cattrs-1.0.0-py2.py3-none-any.whl
Requirement already satisfied: attrs>=17.3 in /usr/local/lib/python3.6/site-packages (from cattrs==1.0.0)
Installing collected packages: cattrs
Found existing installation: cattrs 1.1.0
Uninstalling cattrs-1.1.0:
Successfully uninstalled cattrs-1.1.0
Successfully installed cattrs-1.0.0
[xxx@xxx ~]# airflow initdb
DB: sqlite:////root/airflow/airflow.db
[2020-10-30 09:36:05,431] {db.py:378} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/usr/local/lib/python3.6/site-packages/alembic/ddl/sqlite.py:44: UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
...
Done.
```
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here in side a detail tag:
<details><summary>x.log</summary> lots of stuff </details>
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-721198052
> @jodmoreira oh how i envy you, the constraint file `constraints-3.8.txt` doesn't work for my `1.10.12` 😿 [#8467 (comment)](https://github.com/apache/airflow/issues/8467#issuecomment-720712897)
Just update both `attrs` and `cattrs` that too should work.
or just downgrade `cattrs`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] noelmcloughlin edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
noelmcloughlin edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739439092
> The new version of PIP has MANY problems and broke a lot of other projects.
I never use PyPi pip because OS distributions provide stable `python-pip, python2-pip, python3-pip` packages so I use those. This means NEVER running `pip install --upgrade pip` because that installs PyPi pip ontop distribution python-pip and things breaking horribly when packages get installed in different pip library locations (a great frustration). I'd recommend Airflow adopts the same policy because this philosophy works great for me. I avoid PyPi pip unless I have to use it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dark-spark2 edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
dark-spark2 edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-722344368
@kaxil can I change attrs to higher version( attrs==20.1.0 ) for airflow 1.10.12?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sephethus commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
sephethus commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-738141881
Nothing works in this entire thread, nothing. I'm using docker-compose up to create a local instance and it all fails. 1.10.13, adding cattrs and attrs and upping the version of oauthlib and requests-oauthlib, nothing works, period. What now?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gaj995 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
gaj995 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-725596109
> You can use the following for now, this will be fixed in 1.10.13
>
> ```shell
> pip install apache-airflow==1.10.12 \
> --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"
> ```
Didn't work for me :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-742529511
Are you trying to develop Airflow or run it for "production" on MacOS? We do not support MacOS as "execution" environment" - only Linux systems are supported for 'running' Airflow', MacOS in our case is only supported as development environment.
It looks like a Mac-only thing, so if you are using MacOS to develop airflow and you can use dockerised development environment instead, while we manage to pin-point it? (.
On MacOS we also have Breeze that provides a fully-featured docker-based dev environment that works flawlessly with MacOS as well. We fully support it and there is plenty of documentation and movies on how to use Breeze for development purpose including 'start-airflow' command that automatically spins of running airflow scheduler + webserver + command line from your local sources.
BTW. If you have migrated to BigSur, this might be the root cause of the problem. Usually the "POSIX" development environment has lots of problems with the new releases of MacOS. It usually takes couple of months and few releases of patches + some releases of brew and other dev environments to get it working. Again Breeze is your best bet there as it is "stable" environment. As on as you have Docker and Docker-compose, it will work on MacOS/Linux and even WSL2 for Windows in similar way.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] noelmcloughlin commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
noelmcloughlin commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739440890
> when i use pip3 install cattrs==1.0.0.
Beware of cattrs 1.n.n hell with Python 3.9 (i.e. archlinux).
- https://github.com/Tinche/cattrs/issues/100
- https://github.com/aws/aws-cdk/issues/11043
"ImportError: cannot import name '_Union' from 'typing'"
ImportError: cannot import name 'resolve_types' from 'attr'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-738166144
> Nothing works in this entire thread, nothing. I'm using docker-compose up to create a local instance and it all fails. 1.10.13, adding cattrs and attrs and upping the version of oauthlib and requests-oauthlib, nothing works, period. What now?
There is a known bug with the latest PiP that was released two days ago. You probably hit the problem described here: https://github.com/pypa/pip/issues/9203
The new version of PIP has MANY problems and broke a lot of other projects. For now the best course of action is to downgrade Pip to 20.2.4 version (this is what we did in official docker images and this is what is recommended in 1.10.14 documentation (release candidate will be there shortly soon).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] canerakin111 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
canerakin111 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-723386863
> You can use the following for now, this will be fixed in 1.10.13
>
> ```shell
> pip install apache-airflow==1.10.12 \
> --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.7.txt"
> ```
Thanks a lot!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jesperbruunhansen commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
jesperbruunhansen commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-742386060
Unfortunately nothing is the thread works for me :(
I've tried installing Airflow with the command mentioned by @kaxil, but I still get the error.
The error persists in python3.7 as well as python3.8 for me.
Pip: 20.2.4
Python: 3.8.6
OS: MacOS 10.15.6
Error when running `airflow initdb`:
```
ImportError: cannot import name 'resolve_types' from 'attr' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/attr/__init__.py)
```
I've also tried to install the 1.10.13 and 1.10.14rc's, but the error stills persists
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gaj995 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
gaj995 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726099567
> > Didn't work for me :(
>
> What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?
Here is the error I am getting. I use the pip method that @kaxil posted above.
On Ubantu 20.04.1
Python 3.8.5
`Traceback (most recent call last):
File "/home/gaj995/.local/bin/airflow", line 25, in <module>
from airflow.configuration import conf
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/__init__.py", line 50, in <module>
from airflow.models import DAG # noqa: E402
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/__init__.py", line 21, in <module>
from airflow.models.baseoperator import BaseOperator, BaseOperatorLink # noqa: F401
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1116, in <module>
@attr.s(auto_attribs=True)
AttributeError: module 'attr' has no attribute 's'`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726130970
@gaj995
Since you are using Python 3.8 use the following:
```
pip install apache-airflow==1.10.12 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.8.txt"
```
> > > Didn't work for me :(
> >
> >
> > What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?
>
> Here is the error I am getting. I used the pip method that @kaxil posted above.
>
> On Ubantu 20.04.1
> Python 3.8.5
>
> ```
> `Traceback (most recent call last):
> File "/home/gaj995/.local/bin/airflow", line 25, in <module>
> from airflow.configuration import conf
> File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/__init__.py", line 50, in <module>
> from airflow.models import DAG # noqa: E402
> File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/__init__.py", line 21, in <module>
> from airflow.models.baseoperator import BaseOperator, BaseOperatorLink # noqa: F401
> File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1116, in <module>
> @attr.s(auto_attribs=True)
> AttributeError: module 'attr' has no attribute 's'`
> ```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dark-spark2 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
dark-spark2 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-722344368
@kaxil can I bump:
can I change attrs to higher version( attrs==20.1.0 ) for airflow 1.10.12?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sephethus commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
sephethus commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739998480
> > Nothing works in this entire thread, nothing. I'm using docker-compose up to create a local instance and it all fails. 1.10.13, adding cattrs and attrs and upping the version of oauthlib and requests-oauthlib, nothing works, period. What now?
>
> There is a known bug with the latest PiP that was released two days ago. You probably hit the problem described here: [pypa/pip#9203](https://github.com/pypa/pip/issues/9203)
>
> The new version of PIP has MANY problems and broke a lot of other projects. For now the best course of action is to downgrade Pip to 20.2.4 version (this is what we did in official docker images and this is what is recommended in 1.10.14 documentation (release candidate will be there shortly soon).
According to my pip3 --version I'm on 20.0.2 so that can't be the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vinaymor002 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
vinaymor002 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-734225910
Getting error with python 3.9.0
`ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/xf/hwq0q8rn56xcd_830x9q24c40000gn/T/tmpk9hup6wq
cwd: /private/var/folders/xf/hwq0q8rn56xcd_830x9q24c40000gn/T/pip-install-9jhr51fl/pandas`
Used below command
`pip install apache-airflow==1.10.12 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.8.txt"`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-725613679
> Didn't work for me :(
What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gaj995 edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
gaj995 edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726099567
> > Didn't work for me :(
>
> What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?
Here is the error I am getting. I use the pip method that @kaxil posted above.
On Ubantu 20.04.1
Python 3.8.5
```
`Traceback (most recent call last):
File "/home/gaj995/.local/bin/airflow", line 25, in <module>
from airflow.configuration import conf
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/__init__.py", line 50, in <module>
from airflow.models import DAG # noqa: E402
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/__init__.py", line 21, in <module>
from airflow.models.baseoperator import BaseOperator, BaseOperatorLink # noqa: F401
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1116, in <module>
@attr.s(auto_attribs=True)
AttributeError: module 'attr' has no attribute 's'`
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726149786
Good catch @kaxil!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zachliu commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
zachliu commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-721189925
@jodmoreira oh how i envy you, the constraint file `constraints-3.8.txt` doesn't work for my `1.10.12` :crying_cat_face: https://github.com/apache/airflow/issues/8467#issuecomment-720712897
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-721572179
Yeah. Constraints file does not work if you already have higher version installed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-734260752
> Getting error with python 3.9.0
>
> `ERROR: Command errored out with exit status 1: command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/xf/hwq0q8rn56xcd_830x9q24c40000gn/T/tmpk9hup6wq cwd: /private/var/folders/xf/hwq0q8rn56xcd_830x9q24c40000gn/T/pip-install-9jhr51fl/pandas`
>
> Used below command
> `pip install apache-airflow==1.10.12 --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.8.txt"`
Python 3.9 has known problems with dependencies - we are not supporting it yet. This will happen after 2.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gaj995 edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
gaj995 edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726099567
> > Didn't work for me :(
>
> What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?
Here is the error I am getting. I used the pip method that @kaxil posted above.
On Ubantu 20.04.1
Python 3.8.5
```
`Traceback (most recent call last):
File "/home/gaj995/.local/bin/airflow", line 25, in <module>
from airflow.configuration import conf
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/__init__.py", line 50, in <module>
from airflow.models import DAG # noqa: E402
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/__init__.py", line 21, in <module>
from airflow.models.baseoperator import BaseOperator, BaseOperatorLink # noqa: F401
File "/home/gaj995/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 1116, in <module>
@attr.s(auto_attribs=True)
AttributeError: module 'attr' has no attribute 's'`
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rajnathsah commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
rajnathsah commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-736226617
This issue seems to be fixed in 1.10.13.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] noelmcloughlin commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
noelmcloughlin commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739439092
> The new version of PIP has MANY problems and broke a lot of other projects.
I never use PyPi pip because OS distributions provide stable `python-pip, python2-pip, python3-pip` packages so I use those. This means NEVER running `pip install --upgrade pip` because that installs PyPi pip ontop distribution python-pip and things breaking horribly when packages get installed in different pip library locations (a great fustration). I'd recommend Airflow adopts the same policy.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739528100
> I'd recommend Airflow adopts the same policy because this philosophy works great for me.
Unfortunately, It is beyond of our control when it comes to our users. We have no physical capacity to force our users to use our preferred installation method. So we better try it ourselves and . We have to assume our users installed latest PIP (which the cannot for now as stated in #12838 which is currently. And in many toolchains where Airflow is installed people build Airflow Images automatically and running the PyPI pip. Including our image which is currently pinned to 20.2.4.
> Beware of cattrs 1.n.n hell with Python 3.9 (i.e. archlinux).
Yep. This is one of the reasons why [Python 3.9 is not officially supported yet by Airflow](https://github.com/apache/airflow/blob/master/README.md#requirements)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cgddrd edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
cgddrd edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-719472863
Can confirm we're experiencing this as part of our production stack (auto scaling actions are now failing when trying to install 1.10.12) - a hotfix would be greatly appreciated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jesperbruunhansen edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
jesperbruunhansen edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-742386060
Unfortunately nothing is the thread works for me :(
I've tried installing Airflow with the command mentioned by @kaxil, but I still get the error.
The error persists in python3.7 as well as python3.8 for me.
Pip: 20.2.4
Python: 3.8.6 or 3.7.4
OS: MacOS 10.15.6
Error when running `airflow initdb`:
```
ImportError: cannot import name 'resolve_types' from 'attr' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/attr/__init__.py)
```
I've also tried to install the 1.10.13 and 1.10.14rc's, but the error stills persists
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-725613679
> Didn't work for me :(
What do you mean by 'didn't work"? Can you please provide some specifics on what you tried and how it did not work @gaj995 ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mmgithub123 commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
mmgithub123 commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-723829647
great!
I have the some question，
when i use pip3 install cattrs==1.0.0.
fix my problem
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sephethus edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
sephethus edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-738141881
Nothing works in this entire thread, nothing. I'm using docker-compose up to create a local instance and it all fails. 1.10.13, adding cattrs and attrs and upping the version of oauthlib and requests-oauthlib, nothing works, period. What now?
`ImportError: cannot import name 'resolve_types' from 'attr' (/usr/local/lib/python3.7/site-packages/attr/__init__.py)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] cgddrd commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
cgddrd commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-719472863
Can confirm we're experiencing this as part of our production stack (auto scaling actions are now failing) - a hotfix would be greatly appreciated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-719453425
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-736320056
Yep. And in 1.10.14 we have improved constraints that are not going to be broken (they pass 'pip check;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zachliu commented on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
zachliu commented on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-721881515
@kaxil @potiuk ah, i'm sorry i didn't describe it clearly :sweat_smile:
the constraint file `constraints-3.8.txt` did solve the `attr` and/or `cattrs` issue, but in order to use it i had to do the following changes due to https://github.com/apache/airflow/issues/8467#issuecomment-720712897:
original:
```
oauthlib==3.1.0
requests-oauthlib==1.3.0
```
my updates:
```
oauthlib==2.1.0
requests-oauthlib==1.1.0
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-726130970
@gaj995
Since you are using Python 3.8 use the following:
```
pip install apache-airflow==1.10.12 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.12/constraints-3.8.txt"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] noelmcloughlin edited a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
noelmcloughlin edited a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-739439092
> The new version of PIP has MANY problems and broke a lot of other projects.
I never use PyPi pip because OS distributions provide stable `python-pip, python2-pip, python3-pip` packages so I use those. This means NEVER running `pip install --upgrade pip` because that installs PyPi pip ontop distribution python-pip and things breaking horribly when packages get installed in different pip library locations (a great frustration). I'd recommend Airflow adopts the same policy because this philosophy works great for me. I avoid PyPi pip unless I have to use it (viurtualenvs).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jesperbruunhansen removed a comment on issue #11965: Airflow fails to initdb with cattrs 1.1.0
Posted by GitBox <gi...@apache.org>.
jesperbruunhansen removed a comment on issue #11965:
URL: https://github.com/apache/airflow/issues/11965#issuecomment-742386060
Unfortunately nothing is the thread works for me :(
I've tried installing Airflow with the command mentioned by @kaxil, but I still get the error.
The error persists in python3.7 as well as python3.8 for me.
Pip: 20.2.4
Python: 3.8.6 or 3.7.4
OS: MacOS 10.15.6
Error when running `airflow initdb`:
```
ImportError: cannot import name 'resolve_types' from 'attr' (/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/attr/__init__.py)
```
I've also tried to install the 1.10.13 and 1.10.14rc's, but the error stills persists
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org