You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/12 19:15:56 UTC

[GitHub] [airflow] pingzh opened a new issue, #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

pingzh opened a new issue, #23679:
URL: https://github.com/apache/airflow/issues/23679

   ### Apache Airflow version
   
   main (development)
   
   ### What happened
   
   trying to run airflow tasks run command locally:
   
   ```
   airflow tasks run example_bash_operator also_run_this scheduled__2022-05-08T00:00:00+00:00  --job-id 237 --local --subdir /Users/ping_zhang/airlab/repos/airflow/airflow/example_dags/example_bash_operator.py -f -i
   ```
   
   However, it always errors out:
   
   
   ```
   [2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this Traceback (most recent call last):
   [2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/miniforge3/envs/apache-***/bin/***", line 33, in <module>
   [2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     sys.exit(load_entry_point('apache-***', 'console_scripts', '***')())
   [2022-05-12 12:08:32,893] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/__main__.py", line 38, in main
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     args.func(args)
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/cli/cli_parser.py", line 51, in command
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     return func(*args, **kwargs)
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/utils/cli.py", line 99, in wrapper
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     return f(*args, **kwargs)
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 369, in task_run
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     ti, _ = _get_ti(task, args.execution_date_or_run_id, args.map_index, pool=args.pool)
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/utils/session.py", line 71, in wrapper
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     return func(*args, session=session, **kwargs)
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 152, in _get_ti
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     dag_run, dr_created = _get_dag_run(
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this   File "/Users/ping_zhang/airlab/repos/***/***/cli/commands/task_command.py", line 112, in _get_dag_run
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this     raise DagRunNotFound(
   [2022-05-12 12:08:32,894] {base_task_runner.py:109} INFO - Job 265: Subtask also_run_this ***.exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of 'scheduled__2022-05-08T00:00:00+00:00' not found
   [2022-05-12 12:08:33,014] {local_task_job.py:163} INFO - Task exited with return code 1
   [2022-05-12 12:08:33,048] {local_task_job.py:265} INFO - 0 downstream tasks scheduled from follow-on schedule check
   [2022-05-12 12:11:30,742] {taskinstance.py:1120} INFO - Dependencies not met for <TaskInstance: example_bash_operator.also_run_this scheduled__2022-05-08T00:00:00+00:00 [running]>, dependency 'Task Instance Not Running' FAILED: Task is in the running state
   [2022-05-12 12:11:30,743] {local_task_job.py:102} INFO - Task is not able to be run
   ```
   
   i have checked the dag_run does exist in my db:
   
   ![image](https://user-images.githubusercontent.com/8662365/168151064-b610f6c9-c9ab-40b9-9b5b-fb7e8773aed9.png)
   
   
   
   
   
   ### What you think should happen instead
   
   _No response_
   
   ### How to reproduce
   
   pull the latest main branch with this commit: `7277122ae62305de19ceef33607f09cf030a3cd4`
   
   run airflow scheduler, webserver and worker locally with `CeleryExecutor`.
   
   ### Operating System
   
   Apple M1 Max, version: 12.2
   
   ### Versions of Apache Airflow Providers
   
   NA
   
   ### Deployment
   
   Other
   
   ### Deployment details
   
   on my local mac with latest main branch, latest commit: `7277122ae62305de19ceef33607f09cf030a3cd4`
   
   
   
   ### Anything else
   
   Python version:
   
   Python 3.9.7
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126661365

   Good diagnosis @pingzh . Confirmed!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1125433623

   the `_start_by_exec` is broken in `StandardTaskRunner`
   
   The `session` was not configed correctly. This is why it always complains `DagRunNotFound`:
   
   ![image](https://user-images.githubusercontent.com/8662365/168170032-6e226844-823d-4fed-96e8-dc7f8d843aee.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1125338508

   the root cause should be https://github.com/apache/airflow/blob/749e53def43055225a2e5d09596af7821d91b4ac/airflow/cli/commands/task_command.py#L106
   
   when the `run_id` is passed in, instead of the `execution_date`, this query uses `None` as the `execution_date` to query dag_run


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126511471

   looks like the bug is introduced in this pr https://github.com/apache/airflow/pull/22284/files.
   
   my `airflow.cfg` does not have `[database]` section, but i have `sql_alchemy_conn` under `core`. so when the StandardTaskRunner creates the tmp cfg file, `sql_alchemy_conn` is in both `[database]` and `[core]`. the value of `sql_alchemy_conn` in `[database]` is the `sqlite`, while `sql_alchemy_conn` in `[core]` has the my intended value, (mysql). then `sql_alchemy_conn` is get from `[database]`, which is why it is `sqlite` instead of `mysql`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] pingzh commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
pingzh commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1133442138

   thanks @potiuk for quickly getting this fixed 👍


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #23679:
URL: https://github.com/apache/airflow/issues/23679#issuecomment-1126660186

   AAARGH.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk closed issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of

Posted by GitBox <gi...@apache.org>.
potiuk closed issue #23679: exceptions.DagRunNotFound: DagRun for example_bash_operator with run_id or execution_date of
URL: https://github.com/apache/airflow/issues/23679


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org