You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "cmarteepants (via GitHub)" <gi...@apache.org> on 2025/07/08 20:17:14 UTC

[I] **Search - Dags**: Filter by import errors, DAG owner, stale DAGs, bundle name, timetable type, unscheduled DAGs (DAGs without a schedule set) [airflow]

cmarteepants opened a new issue, #53041:
URL: https://github.com/apache/airflow/issues/53041

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3117596247

   Assigned!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "cmarteepants (via GitHub)" <gi...@apache.org>.
cmarteepants commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3109576433

   Thanks @AryanK1511 - sorry for the delay. Agree that that's weird. If you come across more that don't work or can't get to any, just comment on the issue and move on. All and any progress is helpful, happy to create other issues if we need to for tracking purposes.
   
   Thanks again for taking this on! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3094705367

   Hi! I had a question and would appreciate your input on this.
   
   You mentioned wanting a UI filter to show only DAGs with import errors (`has_import_errors = true`). That makes sense, and I actually opened [this issue](https://github.com/apache/airflow/issues/53536) to implement the change on the API side as well.
   
   However, after digging into it, I realized there’s a complication: even though the `Dags` table has a `has_import_errors` field (which is `false` when there are no errors), if a DAG *does* have an import error, it never gets an entry in the `Dags` table at all. I tested this by creating a DAG with an import error:
   
   ```python
   from airflow import DAG
   from datetime import datetime
   
   # This will break the import!
   this_will_throw_an_error
   
   default_args = {
       'start_date': datetime(2023, 1, 1),
   }
   
   dag = DAG(
       'broken_dag',
       default_args=default_args,
       schedule_interval=None,
   )
   ```
   
   As expected, the database only has my older (valid) DAG, but not the new one with the import error. 
   
   <img width="900" height="191" alt="Image" src="https://github.com/user-attachments/assets/8f11d5c7-a8ae-4f09-9885-ce1ec59d9aa5" />
   
   The broken DAG does show up in the Airflow UI (in the "Import Errors" section), but there’s no entry for it in the Dags table in the DB. 
   
   <img width="124" height="83" alt="Image" src="https://github.com/user-attachments/assets/2115d883-d3d7-4d02-8951-129f3cc6b683" />
   
   <img width="848" height="208" alt="Image" src="https://github.com/user-attachments/assets/dd59ed07-692d-40a5-b515-97c75ddf0499" />
   
   <img width="946" height="421" alt="Image" src="https://github.com/user-attachments/assets/bafd60c2-79f3-47e8-a431-ec44dc97ec80" />
   
   This seems to be because there’s a separate `import_error` table handling these cases. I’ve also confirmed this with @kalyan.
   
   <img width="858" height="539" alt="Image" src="https://github.com/user-attachments/assets/f0ba340a-f848-49bd-9f2a-a27a00dccf8f" />
   
   Given this setup, it seems like adding an API/UI filter on `has_import_errors` wouldn’t actually be useful, since the broken DAGs aren’t in the `Dags` table to be filtered in the first place, they’re already shown separately in the UI.
   
   Could you please advise whether we should still go ahead with this filter, or if it makes more sense to close the issue I opened yesterday as "not planned"? I can start looking into other filters in the meantime. Would appreciate a quick response, thanks!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3172426300

   > Hi [@vatsrahul1001](https://github.com/vatsrahul1001) , Have completed the work on [#53738](https://github.com/apache/airflow/issues/53738)
   > 
   > Will be starting work on [#53741](https://github.com/apache/airflow/issues/53741)
   
   Thanks for update @shreyaskj-0710.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3109523410

   Yes maybe just disregard that field for now and focus on other filters.
   
   It’s weird I wasn’t able to identify where that Dag attribute was set. I’ll need to do more digging. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3092311498

   Thank you for assigning this to me. I will start working on it now. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "shreyaskj-0710 (via GitHub)" <gi...@apache.org>.
shreyaskj-0710 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3162717758

   Hi,
   
   If any of the issues listed above are unattended, 
   
   I would like to work on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3163137077

   Re-assigned #53740 to @vatsrahul1001 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "gyli (via GitHub)" <gi...@apache.org>.
gyli commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3192556941

   About the question whether `has_import_errors` is helpful that @AryanK1511 raised above, I think it can still be helpful to filter the DAGs that have been successfully loaded at least once before. For those dags, they have records in `dags` table, and if they have any import error later, the `has_import_errors` field will also be updated correctly. 
   
   So I think it's still useful to create such filter. I also suggest to update any related doc, to highlight the `Import Errors` filter only returns DAGs that have been successfully loaded before.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3172420597

   @shreyaskj-0710 If you haven’t started working on the issues below, I can take them up as they are required for the parent issue:
   https://github.com/apache/airflow/issues/53741
   https://github.com/apache/airflow/issues/53738


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3157558607

   @AryanK1511 just checking in to see if you have had a chance to work on the API changes?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "shreyaskj-0710 (via GitHub)" <gi...@apache.org>.
shreyaskj-0710 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3162778368

   Hi @phanikumv , 
   
   Sure I will comment ,
   
   I have currently picked up one of the above listed issue [#53738 ](https://github.com/apache/airflow/issues/53738) as there was no recent activity on the ticket.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3081792996

   I would love to work on this issue. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3115848358

   I have created the following issues for the API changes and once I complete them, I will add the UI filters in one swoop. Can someone please assign these to me:
   
   - #53738
   - #53739
   - #53740
   - #53741


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "phanikumv (via GitHub)" <gi...@apache.org>.
phanikumv commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3162736401

   I can assign #53741 to you @shreyaskj-0710 , can you comment on the ticket?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "shreyaskj-0710 (via GitHub)" <gi...@apache.org>.
shreyaskj-0710 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3172424185

   Hi @vatsrahul1001 ,
   Have completed the work on #53738 
   
   Will be starting work on #53741 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3111679681

   Thank you so much for your help guys. I will start working on the other stuff now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "AryanK1511 (via GitHub)" <gi...@apache.org>.
AryanK1511 commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3170656701

   Something came up, so I won’t be able to work on Airflow for the next couple of weeks. Could this issue be assigned to someone else?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3307733321

   > @pierrejeambrun But aren't "Filter DAGs with Import Errors" and "Filter DAGs by Owner" missing from subissue list?
   
   The way I see it is that we have 'subtasks' for every missing filters on the API. Both `import errors` and `owners` are already available filters on the API side, this is why there is no sub issue specific to this.
   
   This parent ticket https://github.com/apache/airflow/issues/53041 that track the feature completion (back and UI) mentions both `Filter DAGs with Import Errors` and `Filter DAGs by Owner` in addition to other filters and will need their UI implementation.
   
   Does that make sense?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Filters] Add filtering support to Search - Dags [airflow]

Posted by "sfc-gh-dfrail (via GitHub)" <gi...@apache.org>.
sfc-gh-dfrail commented on issue #53041:
URL: https://github.com/apache/airflow/issues/53041#issuecomment-3299402507

   @pierrejeambrun But aren't "Filter DAGs with Import Errors" and "Filter DAGs by Owner" missing from subissue list?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org