You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/17 20:06:05 UTC
[GitHub] [airflow] kn6405 opened a new issue #17662: Dags volume not getting mounted in webserver pod?
kn6405 opened a new issue #17662:
URL: https://github.com/apache/airflow/issues/17662
When I deploy via the helm chart the dags volume gets mounted in worker pods, schedular, etc... but does not get mounted in the webserver pod. Noticed the yaml is coded differently than for workers or the schedular (vs. others) and references version numbers older than 2.0.0. So running 2.1.2 the dag never gets mounted.
https://github.com/apache/airflow/blob/c22ed08ec4d8a9def1f09e74d51eafee83f87e8c/chart/templates/webserver/webserver-deployment.yaml#L147
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kn6405 commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
kn6405 commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900632611
> This is correct behaviour. Dags no longer need to be mounted in Webserver in Airflow 2. Webserver reads everything it needs from Serialized DAGs table and will never read DAGs from the files. This is absolutely expected and correct.
With older versions python errors within dags would be displayed on the UI and I believe where checked against python modules installed within the webserver container. How are these dependencies now checked? What container(s) does it look to have all the proper dependencies?
On my old install 1.x the dags had requirement.txt that was mounted and it installed them at startup in most of the containers.
Just want to make sure I understand the new logic in the 2.x version of airflow. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17662:
URL: https://github.com/apache/airflow/issues/17662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900636540
This is also done through DB - there is a DB "import errors" that is populated by scheduler (and I believe it's always been like that)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900624765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900638058
See this page - it explains everything https://airflow.apache.org/docs/apache-airflow/stable/dag-serialization.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900637436
This has been also possible to not mount dags in few last versions of airflow if you had serialization turned on BTW. Simply in Airlfow 2 Dag Serialization is the only way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kn6405 commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
kn6405 commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900632611
> This is correct behaviour. Dags no longer need to be mounted in Webserver in Airflow 2. Webserver reads everything it needs from Serialized DAGs table and will never read DAGs from the files. This is absolutely expected and correct.
With older versions python errors within dags would be displayed on the UI and I believe where checked against python modules installed within the webserver container. How are these dependencies now checked? What container(s) does it look to have all the proper dependencies?
On my old install 1.x the dags had requirement.txt that was mounted and it installed them at startup in most of the containers.
Just want to make sure I understand the new logic in the 2.x version of airflow. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17662:
URL: https://github.com/apache/airflow/issues/17662#issuecomment-900624765
This is correct behaviour. Dags no longer need to be mounted in Webserver in Airflow 2. Webserver reads everything it needs from Serialized DAGs table and will never read DAGs from the files. This is absolutely expected and correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17662: Dags volume not getting mounted in webserver pod?
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17662:
URL: https://github.com/apache/airflow/issues/17662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org