You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/03/10 12:30:54 UTC

[GitHub] [airflow] potiuk opened a new issue, #30022: In-flight DAG Import Errors Cause Tasks to Fail without Logs

potiuk opened a new issue, #30022:
URL: https://github.com/apache/airflow/issues/30022

   ### Discussed in https://github.com/apache/airflow/discussions/29984
   
   <div type='discussions-op-text'>
   
   <sup>Originally posted by **RNHTTR** March  8, 2023</sup>
   ### Apache Airflow version
   
   2.5.1
   
   ### What happened
   
   Today, if a DAG is running, and it has a top-level `Variable.get()` command, and that Variable is deleted while the DAG is still running, the DAG’s next task(s) will fail with no task logs (assuming the Variable eventually gets re-introduced), because the DAG cannot be parsed.
   
   I suspect this would happen with anything that can cause an import error,  but I’m able to easily reproduce it using Variables.
   
   ### What you think should happen instead
   
   Once the DAG is able to be parsed/imported, any tasks that failed due to this problem should say something along the lines of "Task timed out due to a DAG import or parse error."
   
   ### How to reproduce
   
   1. Create an Airflow Variable with the key "some_variable"
   2. Trigger a DAG that makes a top-level call to `Variable.get("some_variable")`. While the DAG is running, delete the Airflow Variable.
   3. Wait 10-15 minutes or so
   4. Recreate an Airflow Variable with the key "some_variable"
   
   ### Operating System
   
   n/a
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   </div>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #30022: In-flight DAG Import Errors Cause Tasks to Fail without Logs

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #30022:
URL: https://github.com/apache/airflow/issues/30022#issuecomment-1463739723

   From the https://github.com/apache/airflow/discussions/29984. seems that new logging framework might allow to surface such errors in the task logs. Cc: @dstandish 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk closed issue #30022: In-flight DAG Import Errors Cause Tasks to Fail without Logs

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #30022: In-flight DAG Import Errors Cause Tasks to Fail without Logs
URL: https://github.com/apache/airflow/issues/30022


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #30022: In-flight DAG Import Errors Cause Tasks to Fail without Logs

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #30022:
URL: https://github.com/apache/airflow/issues/30022#issuecomment-1463746620

   Ah. I see it's been re-opened already.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org