[jira] [Commented] (AIRFLOW-2036) "Broken Dag" messages are basically useless for troubleshooting

["Daniel Imberman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067020#comment-17067020 ] 

Daniel Imberman commented on AIRFLOW-2036:
------------------------------------------

[~aizhamal] should I transfer this ticket to GH or close? Is someone working on this?

> "Broken Dag" messages are basically useless for troubleshooting
> ---------------------------------------------------------------
>
>                 Key: AIRFLOW-2036
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2036
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: ui
>    Affects Versions: 1.7.1, 1.8.0
>            Reporter: Neil Hanlon
>            Priority: Major
>
> Numerous times we've been plagued with the dreaded "Broken dag: <Some import error here>" messages popping up in our WebServer UI.
> Until last night, we all thought this message meant the airflow webserver was unable to process these dags, and had literally no idea why; because not only could I enter airflow's virtual environment and import the packages it was looking for, but i could trigger dag runs and they'd work perfectly.
>  
> Turns out the Broken Dag messages in the UI can come from any airflow process that happens to kick off and be unable to import packages.
>  
> I suggest Airflow modify the {{import_errors}} table to add a hostname, which would at the very least help with troubleshooting and prevent operators from pulling their hair out trying to figure out why their web server is reporting no errors but somehow getting Broken Dag messages.
>  
> Airflow is "highly distributed", but that doesn't mean much when you can't find out which of your nodes is having an issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)