You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Owen-CH-Leung (via GitHub)" <gi...@apache.org> on 2023/05/17 03:30:12 UTC
[GitHub] [airflow] Owen-CH-Leung opened a new pull request, #31336: Update logging-tasks.rst
Owen-CH-Leung opened a new pull request, #31336:
URL: https://github.com/apache/airflow/pull/31336
Update the output of airflow info command
fixes: #31304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pankajkoti commented on a diff in pull request #31336: Update the output of airflow info command in the doc
Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti commented on code in PR #31336:
URL: https://github.com/apache/airflow/pull/31336#discussion_r1196032948
##########
docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -102,14 +102,127 @@ the example below.
$ airflow info
...
- airflow on PATH: [True]
-
- Executor: [SequentialExecutor]
- Task Logging Handlers: [StackdriverTaskHandler]
- SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
- DAGs Folder: [/root/airflow/dags]
- Plugins Folder: [/root/airflow/plugins]
- Base Log Folder: [/root/airflow/logs]
+ Apache Airflow
+ version | 2.7.0.dev0
+ executor | LocalExecutor
+ task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
+ sql_alchemy_conn | postgresql+psycopg2://postgres:airflow@postgres/airflow
+ dags_folder | /files/dags
+ plugins_folder | /root/airflow/plugins
+ base_log_folder | /root/airflow/logs
+ remote_base_log_folder |
+
+
+ System info
+ OS | Linux
+ architecture | arm
+ uname | uname_result(system='Linux', node='01b78939a27f', release='5.15.49-linuxkit', version='#1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022', machine='aarch64', processor='')
+ locale | ('en_US', 'UTF-8')
+ python_version | 3.8.16 (default, May 3 2023, 09:25:10) [GCC 10.2.1 20210110]
+ python_location | /usr/local/bin/python
+
+
+ Tools info
+ git | git version 2.30.2
+ ssh | OpenSSH_8.4p1 Debian-5+deb11u1, OpenSSL 1.1.1n 15 Mar 2022
+ kubectl | NOT AVAILABLE
+ gcloud | NOT AVAILABLE
+ cloud_sql_proxy | NOT AVAILABLE
+ mysql | mysql Ver 15.1 Distrib 10.5.19-MariaDB, for debian-linux-gnu (aarch64) using EditLine wrapper
+ sqlite3 | 3.34.1 2021-01-20 14:10:07 10e20c0b43500cfb9bbc0eaa061c57514f715d87238f4d835880cd846b9ealt1
+ psql | psql (PostgreSQL) 15.2 (Debian 15.2-1.pgdg110+1)
+
+
+ Paths info
+ airflow_home | /root/airflow
+ system_path | /files/bin/:/opt/airflow/scripts/in_container/bin/:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ python_path | /usr/local/bin:/usr/local/lib/python38.zip:/usr/local/lib/python3.8:/usr/local/lib/python3.8/lib-dynload:/usr/local/lib/python3.8/site-packages:/opt/airflow:/files/dags:/root/airflow/config:
+ | /root/airflow/plugins
+ airflow_on_path | True
+
+
+ Providers info
+ apache-airflow-providers-airbyte | 3.3.0
Review Comment:
Base Airflow does not ship all these providers. Do we want to include this huge list of providers here?
When I create a new `virtualenv` and install `airflow` in it, I see only the below providers:
```
Providers info
apache-airflow-providers-common-sql | 1.4.0
apache-airflow-providers-ftp | 3.3.1
apache-airflow-providers-http | 4.3.0
apache-airflow-providers-imap | 3.1.1
apache-airflow-providers-sqlite | 3.3.2
```
It looks like like this list includes the providers that are additionally installed in your setup but it does not list `all` the providers which I realised e.g. when I tried to check, I found that `apache-airflow-providers-kafka` is not listed here. Additionally, `apache-airflow-providers-yandex` is suspended but we're still showing it here.
So in my opinion, it would be good to curtail this list to only the providers that are shipped with base Airflow.
We can wait for what others have to say.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #31336: Update the output of airflow info command in the doc
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #31336:
URL: https://github.com/apache/airflow/pull/31336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Owen-CH-Leung commented on a diff in pull request #31336: Update the output of airflow info command in the doc
Posted by "Owen-CH-Leung (via GitHub)" <gi...@apache.org>.
Owen-CH-Leung commented on code in PR #31336:
URL: https://github.com/apache/airflow/pull/31336#discussion_r1196105894
##########
docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -102,14 +102,127 @@ the example below.
$ airflow info
...
- airflow on PATH: [True]
-
- Executor: [SequentialExecutor]
- Task Logging Handlers: [StackdriverTaskHandler]
- SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
- DAGs Folder: [/root/airflow/dags]
- Plugins Folder: [/root/airflow/plugins]
- Base Log Folder: [/root/airflow/logs]
+ Apache Airflow
+ version | 2.7.0.dev0
+ executor | LocalExecutor
+ task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
+ sql_alchemy_conn | postgresql+psycopg2://postgres:airflow@postgres/airflow
+ dags_folder | /files/dags
+ plugins_folder | /root/airflow/plugins
+ base_log_folder | /root/airflow/logs
+ remote_base_log_folder |
+
+
+ System info
+ OS | Linux
+ architecture | arm
+ uname | uname_result(system='Linux', node='01b78939a27f', release='5.15.49-linuxkit', version='#1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022', machine='aarch64', processor='')
+ locale | ('en_US', 'UTF-8')
+ python_version | 3.8.16 (default, May 3 2023, 09:25:10) [GCC 10.2.1 20210110]
+ python_location | /usr/local/bin/python
+
+
+ Tools info
+ git | git version 2.30.2
+ ssh | OpenSSH_8.4p1 Debian-5+deb11u1, OpenSSL 1.1.1n 15 Mar 2022
+ kubectl | NOT AVAILABLE
+ gcloud | NOT AVAILABLE
+ cloud_sql_proxy | NOT AVAILABLE
+ mysql | mysql Ver 15.1 Distrib 10.5.19-MariaDB, for debian-linux-gnu (aarch64) using EditLine wrapper
+ sqlite3 | 3.34.1 2021-01-20 14:10:07 10e20c0b43500cfb9bbc0eaa061c57514f715d87238f4d835880cd846b9ealt1
+ psql | psql (PostgreSQL) 15.2 (Debian 15.2-1.pgdg110+1)
+
+
+ Paths info
+ airflow_home | /root/airflow
+ system_path | /files/bin/:/opt/airflow/scripts/in_container/bin/:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ python_path | /usr/local/bin:/usr/local/lib/python38.zip:/usr/local/lib/python3.8:/usr/local/lib/python3.8/lib-dynload:/usr/local/lib/python3.8/site-packages:/opt/airflow:/files/dags:/root/airflow/config:
+ | /root/airflow/plugins
+ airflow_on_path | True
+
+
+ Providers info
+ apache-airflow-providers-airbyte | 3.3.0
Review Comment:
Yeah I got all these providers because I'm fetching the outputs from `breeze start-airflow` which includes everything I guess. I've shortened the outputs and only keep the logging part
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pankajkoti commented on a diff in pull request #31336: Update the output of airflow info command in the doc
Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti commented on code in PR #31336:
URL: https://github.com/apache/airflow/pull/31336#discussion_r1196192100
##########
docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -102,15 +102,19 @@ the example below.
$ airflow info
...
- airflow on PATH: [True]
-
- Executor: [SequentialExecutor]
- Task Logging Handlers: [StackdriverTaskHandler]
- SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
- DAGs Folder: [/root/airflow/dags]
- Plugins Folder: [/root/airflow/plugins]
- Base Log Folder: [/root/airflow/logs]
-
+ Apache Airflow
+ version | 2.7.0.dev0
+ executor | LocalExecutor
+ task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
+ sql_alchemy_conn | postgresql+psycopg2://postgres:airflow@postgres/airflow
+ dags_folder | /files/dags
+ plugins_folder | /root/airflow/plugins
+ base_log_folder | /root/airflow/logs
+ remote_base_log_folder |
+
+ [skipping the remaining outputs for brevity]
+
+The output of ``airflow info`` is huge so we only keep those that are relevant to logging configuration.
Review Comment:
```suggestion
The output of `airflow info` above is truncated to only display the section that pertains to the logging configuration.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #31336: Update the output of airflow info command in the doc
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on PR #31336:
URL: https://github.com/apache/airflow/pull/31336#issuecomment-1550902959
Since this section of the documentation is just about debugging logging configurations, it may be a good idea to only include the first section of the output since the rest of the output is not particularly relevant (and add a note describing the reasoning to this omission)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pankajkoti commented on a diff in pull request #31336: Update the output of airflow info command in the doc
Posted by "pankajkoti (via GitHub)" <gi...@apache.org>.
pankajkoti commented on code in PR #31336:
URL: https://github.com/apache/airflow/pull/31336#discussion_r1196032948
##########
docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -102,14 +102,127 @@ the example below.
$ airflow info
...
- airflow on PATH: [True]
-
- Executor: [SequentialExecutor]
- Task Logging Handlers: [StackdriverTaskHandler]
- SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
- DAGs Folder: [/root/airflow/dags]
- Plugins Folder: [/root/airflow/plugins]
- Base Log Folder: [/root/airflow/logs]
+ Apache Airflow
+ version | 2.7.0.dev0
+ executor | LocalExecutor
+ task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
+ sql_alchemy_conn | postgresql+psycopg2://postgres:airflow@postgres/airflow
+ dags_folder | /files/dags
+ plugins_folder | /root/airflow/plugins
+ base_log_folder | /root/airflow/logs
+ remote_base_log_folder |
+
+
+ System info
+ OS | Linux
+ architecture | arm
+ uname | uname_result(system='Linux', node='01b78939a27f', release='5.15.49-linuxkit', version='#1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022', machine='aarch64', processor='')
+ locale | ('en_US', 'UTF-8')
+ python_version | 3.8.16 (default, May 3 2023, 09:25:10) [GCC 10.2.1 20210110]
+ python_location | /usr/local/bin/python
+
+
+ Tools info
+ git | git version 2.30.2
+ ssh | OpenSSH_8.4p1 Debian-5+deb11u1, OpenSSL 1.1.1n 15 Mar 2022
+ kubectl | NOT AVAILABLE
+ gcloud | NOT AVAILABLE
+ cloud_sql_proxy | NOT AVAILABLE
+ mysql | mysql Ver 15.1 Distrib 10.5.19-MariaDB, for debian-linux-gnu (aarch64) using EditLine wrapper
+ sqlite3 | 3.34.1 2021-01-20 14:10:07 10e20c0b43500cfb9bbc0eaa061c57514f715d87238f4d835880cd846b9ealt1
+ psql | psql (PostgreSQL) 15.2 (Debian 15.2-1.pgdg110+1)
+
+
+ Paths info
+ airflow_home | /root/airflow
+ system_path | /files/bin/:/opt/airflow/scripts/in_container/bin/:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ python_path | /usr/local/bin:/usr/local/lib/python38.zip:/usr/local/lib/python3.8:/usr/local/lib/python3.8/lib-dynload:/usr/local/lib/python3.8/site-packages:/opt/airflow:/files/dags:/root/airflow/config:
+ | /root/airflow/plugins
+ airflow_on_path | True
+
+
+ Providers info
+ apache-airflow-providers-airbyte | 3.3.0
Review Comment:
Base Airflow does not ship all these providers. Do we want to include this huge list of providers here?
When I create a new `virtualenv` and install `airflow` in it, I see only the below providers:
```
Providers info
apache-airflow-providers-common-sql | 1.4.0
apache-airflow-providers-ftp | 3.3.1
apache-airflow-providers-http | 4.3.0
apache-airflow-providers-imap | 3.1.1
apache-airflow-providers-sqlite | 3.3.2
```
It looks like like this list includes the providers that are additionally installed in your setup. Additionally, `apache-airflow-providers-yandex` is suspended but we're still showing it here.
So in my opinion, it would be good to curtail this list to only the providers that are shipped with base Airflow.
We can wait for what others have to say.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Owen-CH-Leung commented on pull request #31336: Update the output of airflow info command in the doc
Posted by "Owen-CH-Leung (via GitHub)" <gi...@apache.org>.
Owen-CH-Leung commented on PR #31336:
URL: https://github.com/apache/airflow/pull/31336#issuecomment-1550938210
> Since this section of the documentation is just about debugging logging configurations, it may be a good idea to only include the first section of the output since the rest of the output is not particularly relevant (and add a note describing the reasoning to this omission)
Sure. I've modified the file to only include the first section and added a short note to explain
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Owen-CH-Leung commented on a diff in pull request #31336: Update the output of airflow info command in the doc
Posted by "Owen-CH-Leung (via GitHub)" <gi...@apache.org>.
Owen-CH-Leung commented on code in PR #31336:
URL: https://github.com/apache/airflow/pull/31336#discussion_r1196212651
##########
docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst:
##########
@@ -102,15 +102,19 @@ the example below.
$ airflow info
...
- airflow on PATH: [True]
-
- Executor: [SequentialExecutor]
- Task Logging Handlers: [StackdriverTaskHandler]
- SQL Alchemy Conn: [sqlite://///root/airflow/airflow.db]
- DAGs Folder: [/root/airflow/dags]
- Plugins Folder: [/root/airflow/plugins]
- Base Log Folder: [/root/airflow/logs]
-
+ Apache Airflow
+ version | 2.7.0.dev0
+ executor | LocalExecutor
+ task_logging_handler | airflow.utils.log.file_task_handler.FileTaskHandler
+ sql_alchemy_conn | postgresql+psycopg2://postgres:airflow@postgres/airflow
+ dags_folder | /files/dags
+ plugins_folder | /root/airflow/plugins
+ base_log_folder | /root/airflow/logs
+ remote_base_log_folder |
+
+ [skipping the remaining outputs for brevity]
+
+The output of ``airflow info`` is huge so we only keep those that are relevant to logging configuration.
Review Comment:
Thanks @pankajkoti . Adopted your suggested changes =D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org