You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/10 21:53:58 UTC
[GitHub] [airflow] dlampsi opened a new issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
dlampsi opened a new issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247
**Apache Airflow version**: 1.10.10
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): -
**Environment**:
- **Cloud provider or hardware configuration**: MacBook Pro (2019), i5, 16gp RAM
- **OS** (e.g. from /etc/os-release): Debian GNU/Linux 9 (stretch)
- **Kernel** (e.g. `uname -a`): Linux 9d5d03cba741 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 GNU/Linux
- **Install tools**: Run Airflow in Docker containers
- **Others**:
**What happened**:
When use CeleryExecutor + Dag Serialization triggering DAGs via Web UI gets error:
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 121, in wrapper
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/decorators.py", line 56, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/www_rbac/views.py", line 1050, in trigger
external_trigger=True
File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/models/dag.py", line 1818, in create_dagrun
return self.get_dag().create_dagrun(run_id=run_id,
AttributeError: 'NoneType' object has no attribute 'create_dagrun'
```
**What you expected to happen**:
DAG can be triggered via Web UI without exception.
**How to reproduce it**:
1. Deploy Airflow Cluster via docker-compose (example below)
2. Using CeleryExecutor in separated Docker containers for webserver, scheduler, and worker
3. Disable mount DAG files on webserver container and configure DAG Serialisation:
```
AIRFLOW__CORE__STORE_SERIALIZED_DAGS=True
AIRFLOW__CORE__MIN_SERIALIZED_DAG_UPDATE_INTERVAL=30
AIRFLOW__CORE__STORE_DAG_CODE=True
```
5. Open Web UI and try to trigger any DAG.
6. Gets error `AttributeError: 'NoneType' object has no attribute 'create_dagrun'`
**Anything else we need to know**:
Used docker-compose.yml (see `NOTE!!` comment):
```yml
version: "3.7"
services:
postgres:
container_name: airflow_postgres
image: postgres:10
environment:
POSTGRES_HOST: "postgres"
POSTGRES_PORT: "5432"
POSTGRES_DB: "airflow"
POSTGRES_USER: "airflow"
POSTGRES_PASSWORD: "airflow"
redis:
container_name: airflow_redis
image: 'redis:5.0.5'
webserver:
container_name: airflow_webserver
image: airflow-dev:latest
depends_on:
- postgres
- redis
env_file:
.env
ports:
- "8080:8080"
command: webserver
# NOTE!! If uncomment mount bellow, trigger dag via web UI works.
#volumes:
#- ./dags:/usr/local/airflow/dags
scheduler:
container_name: airflow_scheduler
image: airflow-dev:latest
depends_on:
- webserver
env_file:
.env
command: scheduler
volumes:
- ./dags:/usr/local/airflow/dags
flower:
container_name: airflow_flower
image: airflow-dev:latest
restart: always
depends_on:
- redis
env_file:
.env
environment:
CELERY_CONFIG_MODULE: airflow.config_templates.default_celery
ports:
- 5555:5555
command: flower
worker:
container_name: airflow_worker
image: airflow-dev:latest
restart: always
ports: ['8793:8793']
depends_on:
- webserver
- scheduler
env_file:
.env
command: worker -q tech
volumes:
- ./dags:/usr/local/airflow/dags
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612644148
You seem to be using `airflow-dev` image not Airflow 1.10.10
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612667456
> > You seem to be using `airflow-dev` image. Are you sure it is Airflow 1.10.10?
>
> Yes, I'm sure. It's my local image name with Airflow 1.10.10.
I can't seem to.reproduce it, can you push your image to docker for me to try
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil edited a comment on issue #8247: Serialized DAG
trigger via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-614970236
Meanwhile, can you try the following docker image and let me know if that solves the issue for you:
```
docker pull kaxil/docker-airflow:1.10.10
```
Only use the non-rbac UI
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil edited a comment on issue #8247: Serialized DAG
trigger via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612644148
You seem to be using `airflow-dev` image. Are you sure it is Airflow 1.10.10?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #8247: Serialized
DAG trigger via UI error - AttributeError: 'NoneType' object has no
attribute 'create_dagrun'
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612234631
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dlampsi commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
dlampsi commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612667217
> You seem to be using `airflow-dev` image. Are you sure it is Airflow 1.10.10?
Yes, I'm sure. It's my local image name with Airflow 1.10.10.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-614970236
Meanwhile, can you try the following docker image and let me know if that solves the issue for you:
```
docker pull kaxil/docker-airflow:1.10.10
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dlampsi commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
dlampsi commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612668326
> > > You seem to be using `airflow-dev` image. Are you sure it is Airflow 1.10.10?
> >
> >
> > Yes, I'm sure. It's my local image name with Airflow 1.10.10.
>
> I can't seem to.reproduce it, can you push your image to docker for me to try
Sorry, I can't push it right now. But here Dockerfile and entry point script of my image - https://gist.github.com/dlampsi/58300857f59d51512e8b6b941a9dc7a2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #8247: Serialized DAG trigger via
UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-613962637
@dlampsi Is your scheduler pod running when you try the trigger?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-614968274
OK, I could reproduce this. I will create a PR to fix this but would be available in Airflow 1.10.11.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612669210
Did you put the following env variables on the Scheduler POD too:
```
AIRFLOW__CORE__STORE_SERIALIZED_DAGS=True
AIRFLOW__CORE__MIN_SERIALIZED_DAG_UPDATE_INTERVAL=30
AIRFLOW__CORE__STORE_DAG_CODE=True
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dlampsi commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
dlampsi commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612672073
> Did you put the following env variables on the Scheduler POD too:
I'm using `.env` file in each container (docker-compose file example above in issue body).
Here `.env` file content:
```
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=airflow
POSTGRES_USER=airflow
POSTGRES_PASSWORD=airflow
AIRFLOW__CORE__FERNET_KEY=Nk09IXzSw5GGYOUcbey4xxIzeKViJKxq47MqKFPRuik=
AIRFLOW__CORE__COLORED_CONSOLE_LOG=False
AIRFLOW__CORE__LOAD_EXAMPLES=False
AIRFLOW__CORE__EXECUTOR=CeleryExecutor
AIRFLOW__WEBSERVER__BASE_URL=http://127.0.0.1:8080
AIRFLOW__WEBSERVER__ACCESS_LOGFILE=/usr/local/airflow/webserver_access.log
AIRFLOW__WEBSERVER__RBAC=True
RBAC_USER_REGISTRATION_ROLE=Admin
RBAC_LDAP_SERVER=...
RBAC_LDAP_SEARCH=...
RBAC_LDAP_BIND_USER=...
RBAC_LDAP_BIND_PASSWORD=...
RBAC_LDAP_UID_FIELD=sAMAccountName
# Dag Serialization
AIRFLOW__CORE__STORE_SERIALIZED_DAGS=True
AIRFLOW__CORE__MIN_SERIALIZED_DAG_UPDATE_INTERVAL=30
AIRFLOW__CORE__STORE_DAG_CODE=True
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dlampsi commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
dlampsi commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-613965121
> @dlampsi Is your scheduler pod running when you try the trigger?
Yes of course
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8247: Serialized DAG trigger
via UI error - AttributeError: 'NoneType' object has no attribute
'create_dagrun'
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8247: Serialized DAG trigger via UI error - AttributeError: 'NoneType' object has no attribute 'create_dagrun'
URL: https://github.com/apache/airflow/issues/8247#issuecomment-612644698
We had this issue on Airflow Master which was resolved by https://github.com/apache/airflow/pull/8148
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services