[GitHub] [airflow] RNHTTR opened a new issue, #29983: In-flight DAG Import Errors Cause Tasks to Fail without Logs

["RNHTTR (via GitHub)" <gi...@apache.org>]

RNHTTR opened a new issue, #29983:
URL: https://github.com/apache/airflow/issues/29983

   ### Apache Airflow version
   
   2.5.1
   
   ### What happened
   
   Today, if a DAG is running, and it has a top-level `Variable.get()` command, and that Variable is deleted while the DAG is still running, the DAG’s next task(s) will fail with no task logs (assuming the Variable eventually gets re-introduced), because the DAG cannot be parsed.
   
   I suspect this would happen with anything that can cause an import error,  but I’m able to easily reproduce it using Variables.
   
   ### What you think should happen instead
   
   Once the DAG is able to be parsed/imported, any tasks that failed due to this problem should say something along the lines of "Task timed out due to a DAG import or parse error."
   
   ### How to reproduce
   
   1. Create an Airflow Variable with the key "some_variable"
   2. Trigger a DAG that makes a top-level call to `Variable.get("some_variable")`. While the DAG is running, delete the Airflow Variable.
   3. Wait 10-15 minutes or so
   4. Recreate an Airflow Variable with the key "some_variable"
   
   ### Operating System
   
   n/a
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #29983: In-flight DAG Import Errors Cause Tasks to Fail without Logs

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk commented on issue #29983:
URL: https://github.com/apache/airflow/issues/29983#issuecomment-1460615559

   This is expected behaviour. It's inherent in the way how currently DAG folder is used as "current" source of truth even if the structure of the DAG effectively changed in the meantime when task is being executed using the previous structure. This is only one of many manifestations of it.
   
   Until we discuss, approve and implement [DAG versioning](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-36+DAG+Versioning) that is. Any help, initiaive in defining, discussin and implementing it is most welcome.
   
   There are multiple other cases where such breaking changes in DAG can cause more problems like that, solving (or masking) just one of those those problems makes very little sense. we need to make DAG versioning work to allow users to not care about their DAGs being parseable. It makes the problem even worse because people might get used to the fact that the DAGs starts reacting on being modified along the way.
   
   Right now the solution is "Don't change the structure of the DAG in breaking ways"  (including making dynamic changes that will cause different - missing - structure when parsed at different times).
   
   Converting it into discussion in case more discussion is needed, but IMHO there is not much point in raising this and similar issues - we know it works like this  - this is the current  approach that is "good enough" for most cases, and DAG versioning should address the versioning issue "holistically".
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #29983: In-flight DAG Import Errors Cause Tasks to Fail without Logs

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk closed issue #29983: In-flight DAG Import Errors Cause Tasks to Fail without Logs
URL: https://github.com/apache/airflow/issues/29983


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org