You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/12 19:15:56 UTC
[GitHub] [airflow] pingzh opened a new issue, #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
pingzh opened a new issue, #23679:
URL: https://github.com/apache/airflow/issues/23679
### Apache Airflow version
main (development)
### What happened
trying to run airflow tasks run command locally:
```
airflow tasks run example_bash_operator also_run_this scheduled__2022-05-08T00:00:00+00:00 --job-id 237 --local --subdir /Users/ping_zhang/airlab/repos/airflow/airflow/example_dags/example_bash_operator.py -f -i
```
However, it always errors out:
```
[2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this Traceback (most recent call last):
[2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/miniforge3/envs/apache-***/bin/***", line 33, in <module>
[2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this sys.exit(load_entry_point('apache-***', 'console_scripts', '***')())
[2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/__main__.py", line 38, in main
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this args.func(args)
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/cli/cli_parser.py", line 51, in command
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this return func(*args, **kwargs)
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/utils/cli.py", line 99, in wrapper
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this return f(*args, **kwargs)
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 369, in task_run
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this ti, _ = _get_ti(task, args.execution_date_or_run_id, args.map_index, pool=args.pool)
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/utils/session.py", line 71, in wrapper
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this return func(*args, session=session, **kwargs)
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 152, in _get_ti
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this dag_run, dr_created = _get_dag_run(
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 112, in _get_dag_run
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this raise DagRunNotFound(
[2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this ***.exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of 'scheduled__2022-05-08T00:00:00+00:00' not found
[2022-05-12 12:08:33,014] {local_task_job.py:163} INFO - Task exited with return code 1
[2022-05-12 12:08:33,048] {local_task_job.py:265} INFO - 0 downstream tasks scheduled from follow-on schedule check
[2022-05-12 12:11:30,742] {taskinstance.py:1120} INFO - Dependencies not met for <TaskInstance: example_bash_operator.also_run_this scheduled__2022-05-08T00:00:00+00:00 [running]>, dependency 'Task Instance Not Running' FAILED: Task is in the running state
[2022-05-12 12:11:30,743] {local_task_job.py:102} INFO - Task is not able to be run
```
i have checked the dag_run does exist in my db:

### What you think should happen instead
_No response_
### How to reproduce
pull the latest main branch with this commit: `7277122ae62305de19ceef33607f09cf030a3cd4`
run airflow scheduler, webserver and worker locally with `CeleryExecutor`.
### Operating System
Apple M1 Max, version: 12.2
### Versions of Apache Airflow Providers
NA
### Deployment
Other
### Deployment details
on my local mac with latest main branch, latest commit: `7277122ae62305de19ceef33607f09cf030a3cd4`
### Anything else
Python version:
Python 3.9.7
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126661365
Good diagnosis @pingzh . Confirmed!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1125433623
the `_start_by_exec` is broken in `StandardTaskRunner`
The `session` was not configed correctly. This is why it always complains `DagRunNotFound`:

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1125338508
the root cause should be https://github.com/apache/airflow/blob/749e53def43055225a2e5d09596af7821d91b4ac/airflow/cli/commands/task_command.py#L106
when the `run_id` is passed in, instead of the `execution_date`, this query uses `None` as the `execution_date` to query dag_run
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126511471
looks like the bug is introduced in this pr https://github.com/apache/airflow/pull/22284/files.
my `airflow.cfg` does not have `[database]` section, but i have `sql_alchemy_conn` under `core`. so when the StandardTaskRunner creates the tmp cfg file, `sql_alchemy_conn` is in both `[database]` and `[core]`. the value of `sql_alchemy_conn` in `[database]` is the `sqlite`, while `sql_alchemy_conn` in `[core]` has the my intended value, (mysql). then `sql_alchemy_conn` is get from `[database]`, which is why it is `sqlite` instead of `mysql`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1133442138
thanks @potiuk for quickly getting this fixed 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126660186
AAARGH.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
URL: https://github.com/apache/airflow/issues/23679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org