You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/24 15:43:34 UTC
[GitHub] [airflow] honarkhah opened a new issue #12590: Run webserver without installing dag dependencies
honarkhah opened a new issue #12590:
URL: https://github.com/apache/airflow/issues/12590
**Description**
In order to have the webserver easy to maintain and scale it is nice if we can ignore dag dependencies (ex: python libraries) on the webserver and only install them on the worker.
currently webserver throwing an error `{dagbag.py:259} ERROR - Failed to import`
**Use case / motivation**
I want to have the webserver totally independent of what needs to be done! from my perspective only workers need to have all the dependencies installed, also the scheduler could ignore them!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] honarkhah edited a comment on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
honarkhah edited a comment on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733086622
is the v2.0.0 stable now? I had a bad experience with serialized dag in < v1.10.12
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] honarkhah commented on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
honarkhah commented on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733086622
is the v2.0.0 stable now? I had a bad experience with serialized dag in < v1.10.13
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733058063
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb closed issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
ashb closed issue #12590:
URL: https://github.com/apache/airflow/issues/12590
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733106450
From our testing, yes. It's not yet released "GA", but it's getting very close! If you have time to test the beta releases (latest is beta3) that would be helpful!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733068610
2.0.0 does this already, as the Webserver now relies on Dag Serialization and reads things from the DB only.
Easiest issue fix ever -- cos it's already done :grin:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12590: Run webserver without installing dag dependencies
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12590:
URL: https://github.com/apache/airflow/issues/12590#issuecomment-733069090
The scheduler will need them too -- as it "parses" the dag files. The way to avoid needing them on the scheduler is to not have imports at the top-level that are only needed inside execute functions
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org