You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/08 07:51:17 UTC
[GitHub] [airflow-site] ohadmata opened a new pull request #390: Update _index.md
ohadmata opened a new pull request #390:
URL: https://github.com/apache/airflow-site/pull/390
Add simple dag editor plugin
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-site] ohadmata commented on a change in pull request #390: Add simple-dag-editor plugin
Posted by GitBox <gi...@apache.org>.
ohadmata commented on a change in pull request #390:
URL: https://github.com/apache/airflow-site/pull/390#discussion_r593909163
##########
File path: landing-pages/site/content/en/ecosystem/_index.md
##########
@@ -86,3 +86,5 @@ If you would you like to be included on this page, please reach out to the [Apac
[Pylint-Airflow](https://github.com/BasPH/pylint-airflow) - A Pylint plugin for static code analysis on Airflow code.
[whirl](https://github.com/godatadriven/whirl) - Fast iterative local development and testing of Apache Airflow workflows.
+
+[simple-dag-editor](https://github.com/ohadmata/simple-dag-editor) - Zero configuration Airflow plugin that let you manage your DAG files.
Review comment:
@turbaszek
* I released new version today (0.1.0) that add a support for Airflow 2.0+
* The plugin just edit the DAG file itself, it's not related to the Airflow executors.
* The plugin is designed for managing dag files that mounted to the airflow-webserver at "dag_folder".
In the future I want to add S3 support, for git sync you have the code-editor plugin:
https://pypi.org/project/airflow-code-editor
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-site] ohadmata commented on a change in pull request #390: Add simple-dag-editor plugin
Posted by GitBox <gi...@apache.org>.
ohadmata commented on a change in pull request #390:
URL: https://github.com/apache/airflow-site/pull/390#discussion_r595974699
##########
File path: landing-pages/site/content/en/ecosystem/_index.md
##########
@@ -86,3 +86,5 @@ If you would you like to be included on this page, please reach out to the [Apac
[Pylint-Airflow](https://github.com/BasPH/pylint-airflow) - A Pylint plugin for static code analysis on Airflow code.
[whirl](https://github.com/godatadriven/whirl) - Fast iterative local development and testing of Apache Airflow workflows.
+
+[simple-dag-editor](https://github.com/ohadmata/simple-dag-editor) - Zero configuration Airflow plugin that let you manage your DAG files.
Review comment:
@turbaszek your remarks added to the documentation, thanks!
### Use-cases
* This plugin has been developed for a native deployment, where the Airflow webserver & scheduler are sharing the same dag_folder directory (as a mount).
* The plugin is not designed for deployments that synchronize the dag_folder from a remote source (Git/S3)
* You have the ability to disable the plugin for production env (see conf section)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-site] turbaszek merged pull request #390: Add simple-dag-editor plugin
Posted by GitBox <gi...@apache.org>.
turbaszek merged pull request #390:
URL: https://github.com/apache/airflow-site/pull/390
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-site] turbaszek commented on a change in pull request #390: Add simple-dag-editor plugin
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #390:
URL: https://github.com/apache/airflow-site/pull/390#discussion_r593906892
##########
File path: landing-pages/site/content/en/ecosystem/_index.md
##########
@@ -86,3 +86,5 @@ If you would you like to be included on this page, please reach out to the [Apac
[Pylint-Airflow](https://github.com/BasPH/pylint-airflow) - A Pylint plugin for static code analysis on Airflow code.
[whirl](https://github.com/godatadriven/whirl) - Fast iterative local development and testing of Apache Airflow workflows.
+
+[simple-dag-editor](https://github.com/ohadmata/simple-dag-editor) - Zero configuration Airflow plugin that let you manage your DAG files.
Review comment:
Interesting one! Few questions:
- does it work with Airflow 2.0+?
- does it work with all executors?
- if someone is using synchronisation mechanism for syncing DAGs - which version will be used?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-site] turbaszek commented on a change in pull request #390: Add simple-dag-editor plugin
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #390:
URL: https://github.com/apache/airflow-site/pull/390#discussion_r593914447
##########
File path: landing-pages/site/content/en/ecosystem/_index.md
##########
@@ -86,3 +86,5 @@ If you would you like to be included on this page, please reach out to the [Apac
[Pylint-Airflow](https://github.com/BasPH/pylint-airflow) - A Pylint plugin for static code analysis on Airflow code.
[whirl](https://github.com/godatadriven/whirl) - Fast iterative local development and testing of Apache Airflow workflows.
+
+[simple-dag-editor](https://github.com/ohadmata/simple-dag-editor) - Zero configuration Airflow plugin that let you manage your DAG files.
Review comment:
@ohadmata thanks for explanation!
> The plugin just edit the DAG file itself, it's not related to the Airflow executors.
It's important point to me. In most production deployments DAGs are either synchronised from external source (repository, bucket, etc). So my question is what will happen when users edit the DAG via the plugin? Will this change be overridden by some synchronisation proces?
> The plugin is designed for managing dag files that mounted to the airflow-webserver at "dag_folder".
Will this work with CeleryExecutor where webserver and workers are on separate nodes (pods, VMs etc) and do not share same volume (each node has it own volume which is synchronised with for example S3 bucket)?
As a user I miss answers to these questions. So I think it would be a good idea to add some explanation in documentation of your plugin so users are aware of some potential limitations or required setup of their deployment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org