You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "MuthuVignesh91 (via GitHub)" <gi...@apache.org> on 2023/03/02 06:18:20 UTC
[GitHub] [airflow] MuthuVignesh91 opened a new issue, #29864: airflow dags list is not listing all the dags present in airflow home location
MuthuVignesh91 opened a new issue, #29864:
URL: https://github.com/apache/airflow/issues/29864
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
v2.0.2
When trying the "airflow dags list", via the aws mwaa API, we see that the list of dags we receive is not giving complete details.
I dont see any limit option to choose like list-jobs as well.
Unable to trace the exact reason why we are not able to see the complete list of dags.
### What you think should happen instead
_No response_
### How to reproduce
Just perform "airflow dags list" command when there are around 100+ dags in your airflow
### Operating System
Amazon Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Amazon (AWS) MWAA
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell closed issue #29864: airflow dags list is not listing all the dags present in airflow home location
Posted by "josh-fell (via GitHub)" <gi...@apache.org>.
josh-fell closed issue #29864: airflow dags list is not listing all the dags present in airflow home location
URL: https://github.com/apache/airflow/issues/29864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #29864: airflow dags list is not listing all the dags present in airflow home location
Posted by "josh-fell (via GitHub)" <gi...@apache.org>.
josh-fell commented on issue #29864:
URL: https://github.com/apache/airflow/issues/29864#issuecomment-1451943639
Hi @MuthuVignesh91, I cannot reproduce this on Airflow 2.5.1 nor 2.4.0.
DAG used:
```py
from pendulum import datetime
from airflow.decorators import dag
from airflow.operators.empty import EmptyOperator
for i in range(200):
@dag(dag_id=f"dag_{i}", start_date=datetime(2023, 1, 1), schedule=None)
def _dag():
EmptyOperator(task_id="empty")
_dag()
```
DAGs in Airflow UI
<img width="281" alt="image" src="https://user-images.githubusercontent.com/48934154/222450259-9187f29e-4456-438a-a30a-ac9288762716.png">
Running `airflow dags list`
```sh
astro@d2c01fe9b20e:/usr/local/airflow$ airflow dags list | tail -n +3 | head -n -1 | wc -l
200
```
(^ Starting the count at line 3 since there are 2 header rows in the output and there is an empty line at the end.)
Is there always a consistent number of DAGs returned when running this against an MWAA environment? Also, can you try running `airflow dags list` on an environment with a more recent Airflow version too please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #29864: airflow dags list is not listing all the dags present in airflow home location
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #29864:
URL: https://github.com/apache/airflow/issues/29864#issuecomment-1451357666
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] MuthuVignesh91 commented on issue #29864: airflow dags list is not listing all the dags present in airflow home location
Posted by "MuthuVignesh91 (via GitHub)" <gi...@apache.org>.
MuthuVignesh91 commented on issue #29864:
URL: https://github.com/apache/airflow/issues/29864#issuecomment-1457618633
Thanks for responding @josh-fell. I am using v2.0.2 for the same. Also I am using the managed version of airflow which is MWAA. Let me work with the AWS team and come back here if its an airflow issue or some kind of integration/config failure.
We could close this ticket for the moment. You can decide
p.s. Sorry for the delayed response
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #29864: airflow dags list is not listing all the dags present in airflow home location
Posted by "josh-fell (via GitHub)" <gi...@apache.org>.
josh-fell commented on issue #29864:
URL: https://github.com/apache/airflow/issues/29864#issuecomment-1458191734
Sounds good. No problem to reopen later if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org