You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/04/27 12:54:54 UTC
[GitHub] [airflow] tirkarthi opened a new issue, #23283: interactive option is not passed to executor during task run command
tirkarthi opened a new issue, #23283:
URL: https://github.com/apache/airflow/issues/23283
### Apache Airflow version
2.3.0b1 (pre-release)
### What happened
When `--interactive` option is passed during task run command in executor mode the new task run command added to queue doesn't pass `--interactive` causing debugging output to be lost. Example is below case where `invalid_pool` is passed and no debug output is displayed which has the correct error message.
> PYTHONWARNINGS=ignore airflow tasks run simple_print print_date backfill__2022-03-05T00:00:00+00:00 --pool invalid_pool --subdir DAGS_FOLDER/example_bash_4.py --interactive
[2022-04-27 18:13:54,475] {dagbag.py:507} INFO - Filling up the DagBag from /home/karthikeyan/airflow/dags/example_bash_4.py
[2022-04-27 18:13:54,498] {task_command.py:370} INFO - Running <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]> on host laptop
[2022-04-27 18:13:54,501] {executor_loader.py:106} INFO - Loaded executor: SequentialExecutor
Sending to executor.
[2022-04-27 18:13:54,502] {base_executor.py:88} INFO - Adding to queue: ['airflow', 'tasks', 'run', 'simple_print', 'print_date', 'backfill__2022-03-05T00:00:00+00:00', '--local', '--pool', 'invalid_pool', '--subdir', 'DAGS_FOLDER/example_bash_4.py']
[2022-04-27 18:13:54,502] {sequential_executor.py:59} INFO - Executing command: ['airflow', 'tasks', 'run', 'simple_print', 'print_date', 'backfill__2022-03-05T00:00:00+00:00', '--local', '--pool', 'invalid_pool', '--subdir', 'DAGS_FOLDER/example_bash_4.py']
[2022-04-27 18:13:55,236] {dagbag.py:507} INFO - Filling up the DagBag from /home/karthikeyan/airflow/dags/example_bash_4.py
[2022-04-27 18:13:55,258] {task_command.py:370} INFO - Running <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]> on host laptop
Debug output printed that pool is invalid when `interactive` option is passed during command creation.
> PYTHONWARNINGS=ignore airflow tasks run simple_print print_date backfill__2022-03-05T00:00:00+00:00 --pool invalid_pool --subdir DAGS_FOLDER/example_bash_4.py --interactive
[2022-04-27 18:18:06,441] {dagbag.py:507} INFO - Filling up the DagBag from /home/karthikeyan/airflow/dags/example_bash_4.py
[2022-04-27 18:18:06,481] {task_command.py:371} INFO - Running <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]> on host laptop
[2022-04-27 18:18:06,482] {executor_loader.py:106} INFO - Loaded executor: SequentialExecutor
Sending to executor.
[2022-04-27 18:18:06,483] {base_executor.py:88} INFO - Adding to queue: ['airflow', 'tasks', 'run', 'simple_print', 'print_date', 'backfill__2022-03-05T00:00:00+00:00', '--local', '--pool', 'invalid_pool', '--subdir', 'DAGS_FOLDER/example_bash_4.py', '--interactive']
[2022-04-27 18:18:06,484] {sequential_executor.py:59} INFO - Executing command: ['airflow', 'tasks', 'run', 'simple_print', 'print_date', 'backfill__2022-03-05T00:00:00+00:00', '--local', '--pool', 'invalid_pool', '--subdir', 'DAGS_FOLDER/example_bash_4.py', '--interactive']
[2022-04-27 18:18:07,093] {dagbag.py:507} INFO - Filling up the DagBag from /home/karthikeyan/airflow/dags/example_bash_4.py
[2022-04-27 18:18:07,111] {task_command.py:371} INFO - Running <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]> on host laptop
[2022-04-27 18:18:07,171] {taskinstance.py:1164} INFO - Dependencies all met for <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]>
[2022-04-27 18:18:07,173] {taskinstance.py:1154} INFO - Dependencies not met for <TaskInstance: simple_print.print_date backfill__2022-03-05T00:00:00+00:00 [None]>, dependency 'Pool Slots Available' FAILED: Tasks using non-existent pool 'invalid_pool' will not be scheduled
[2022-04-27 18:18:07,175] {taskinstance.py:1346} WARNING -
[2022-04-27 18:18:07,175] {taskinstance.py:1347} WARNING - Rescheduling due to concurrency limits reached at task runtime. Attempt 10 of 11. State set to NONE.
[2022-04-27 18:18:07,175] {taskinstance.py:1354} WARNING -
[2022-04-27 18:18:07,183] {local_task_job.py:101} INFO - Task is not able to be run
### What you think should happen instead
`interactive` option should be passed to the executor during task run command
### How to reproduce
_No response_
### Operating System
Ubuntu 20.04
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #23283: interactive option is not passed to executor during task run command
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #23283: interactive option is not passed to executor during task run command
URL: https://github.com/apache/airflow/issues/23283
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan commented on issue #23283: interactive option is not passed to executor during task run command
Posted by GitBox <gi...@apache.org>.
chenglongyan commented on issue #23283:
URL: https://github.com/apache/airflow/issues/23283#issuecomment-1150212618
The interactive option is a CLI parameter. Why should it be passed to the executor?
https://github.com/apache/airflow/blob/047a6162b0b4cbf07fe2fd978e335839a7d3900b/airflow/cli/commands/task_command.py#L379-L383
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org