[GitHub] [airflow] potiuk commented on pull request #28822: Disable GitSync + Persistence combo in the Helm Chart

[GitBox <gi...@apache.org>]

potiuk commented on PR #28822:
URL: https://github.com/apache/airflow/pull/28822#issuecomment-1376884606

   We had a lot of discussions about it with @jedcunningham in related issues and some time ago I wrote the article describing in detail why git-sync and networked persistence do not play along together (and do not serve any useful use case).
   
   https://medium.com/apache-airflow/shared-volumes-in-airflow-the-good-the-bad-and-the-ugly-22e9f681afca
   
   I think we've accumulated enough of the issues from our users that confirm that they REALLY did not want to use persistence and git-sync, and we had a lot of problems which we could attribute to lack of atomicity that is amplified by git-sync + network persistence.
   
   The last straw for me was this discussion  https://apache-airflow.slack.com/archives/C027H098M1C/p1673333515991549. where the users wanted to use dags.persistence and dags.gitSync to combine development and production workflows in one (which would not work but the user was unaware of that as they did not know that how git-sync works under the hood it would overwrote the manually updated/copied dags).
   
   All-in-all - I think allowing this combo is actively harmful  and we should help our users to make the right decision  (and ourselves by not having similar discussions/getting issues created) by disabling the combo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org