You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/15 20:42:36 UTC
[GitHub] [airflow] fancyfreeman opened a new issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
fancyfreeman opened a new issue #14249:
URL: https://github.com/apache/airflow/issues/14249
**Apache Airflow version: 2.0.1**
**What happened:**
Running an airflow dags test or backfill CLI command show in tutorial, produces the
**dags test cli command result：**
(airflow_venv) (base) app@lunar_01:airflow$ airflow dags test tutorial 2015-06-01
[2021-02-16 04:29:22,355] {dagbag.py:448} INFO - Filling up the DagBag from /home/app/Lunar/src/airflow/dags
[2021-02-16 04:29:22,372] {example_kubernetes_executor_config.py:174} WARNING - Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'
[2021-02-16 04:29:22,373] {example_kubernetes_executor_config.py:175} WARNING - Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']
Traceback (most recent call last):
File "/home/app/airflow_venv/bin/airflow", line 10, in <module>
sys.exit(main())
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/cli.py", line 89, in wrapper
return f(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py", line 389, in dag_test
dag.run(executor=DebugExecutor(), start_date=args.execution_date, end_date=args.execution_date)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/models/dag.py", line 1706, in run
job.run()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 237, in run
self._execute()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 805, in _execute
session=session,
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 715, in _execute_for_run_dates
tis_map = self._task_instances_for_dag_run(dag_run, session=session)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 359, in _task_instances_for_dag_run
dag_run.refresh_from_db()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/models/dagrun.py", line 178, in refresh_from_db
DR.run_id == self.run_id,
File "/home/app/airflow_venv/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3500, in one
raise orm_exc.NoResultFound("No row was found for one()")
sqlalchemy.orm.exc.NoResultFound: No row was found for one()
**backfill cli command result:**
(airflow_venv) (base) app@lunar_01:airflow$ airflow dags backfill tutorial --start-date 2015-06-01 --end-date 2015-06-07
/home/app/airflow_venv/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py:62 PendingDeprecationWarning: --ignore-first-depends-on-past is deprecated as the value is always set to True
[2021-02-16 04:30:16,979] {dagbag.py:448} INFO - Filling up the DagBag from /home/app/Lunar/src/airflow/dags
[2021-02-16 04:30:16,996] {example_kubernetes_executor_config.py:174} WARNING - Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'
[2021-02-16 04:30:16,996] {example_kubernetes_executor_config.py:175} WARNING - Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']
Traceback (most recent call last):
File "/home/app/airflow_venv/bin/airflow", line 10, in <module>
sys.exit(main())
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/cli.py", line 89, in wrapper
return f(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/cli/commands/dag_command.py", line 116, in dag_backfill
run_backwards=args.run_backwards,
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/models/dag.py", line 1706, in run
job.run()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 237, in run
self._execute()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 805, in _execute
session=session,
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 715, in _execute_for_run_dates
tis_map = self._task_instances_for_dag_run(dag_run, session=session)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/jobs/backfill_job.py", line 359, in _task_instances_for_dag_run
dag_run.refresh_from_db()
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/utils/session.py", line 65, in wrapper
return func(*args, session=session, **kwargs)
File "/home/app/airflow_venv/lib/python3.7/site-packages/airflow/models/dagrun.py", line 178, in refresh_from_db
DR.run_id == self.run_id,
File "/home/app/airflow_venv/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3500, in one
raise orm_exc.NoResultFound("No row was found for one()")
sqlalchemy.orm.exc.NoResultFound: No row was found for one()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fancyfreeman commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
fancyfreeman commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-780719442
This is my executor configuration in airflow.cfg :
executor = LocalExecutor
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
vikramkoka commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-780186201
Quick question: Have you tried installing the kubernetes provider?
This is mentioned in the log file above: pip install apache-airflow['cncf.kubernetes']
With Airflow 2.0, there is an explicit separation between Airflow core and Airflow providers from the installation standpoint.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 edited a comment on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 edited a comment on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-803791338
It did not occur when using PostgreSQL as a DB(`sql_alchemy_conn`, `result_backend`).
It seems to occur only in MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kevsjh commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
kevsjh commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-865703609
Is this resolved? I am still seeing the same issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-779439133
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 edited a comment on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 edited a comment on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-865706518
@kevsjh ,
In my case, when Redis was newly configured, the problem did not occur with MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-800036386
I am going through the same issue.
In my case
* executor = CeleryExecutor
* sql_alchemy_conn = mysql (version 8)
* result_backend = mysql (version 8)
* broker_url = redis
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-865706518
@kevsjh ,
In my case, when redis was newly configured, the problem did not occur with MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 edited a comment on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 edited a comment on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-800036386
I am going through the same issue.
In my case
* executor = CeleryExecutor
* sql_alchemy_conn = mysql (version 8.0.23)
* result_backend = mysql (version 8.0.23)
* broker_url = redis (version 5.0.10)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-803791338
It did not occur when using PostgreSQL as a DB.
It seems to occur only in MySQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kevsjh commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
kevsjh commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-866756616
@Qiuzhuang Setting the default_timezone=utc works.
Not sure why when default_timezone is set to Asia, the dag test does not work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fancyfreeman commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
fancyfreeman commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-780423375
I just run airflow Airflow locally. I did as the [quick start](http://airflow.apache.org/docs/apache-airflow/stable/start/local.html#) say.
Do I have to install kubernetes provider ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Qiuzhuang commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
Qiuzhuang commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-865972639
> Is this resolved? I am still seeing the same issue
what is the default_timezone in your airflow.cfg? The default value is
default_timezone = utc
and if you changed this to other timezone like Asia/Shanghai, then backfill/test seems not working.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka commented on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
vikramkoka commented on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-780690135
Oh, in that case I misunderstood.
What executor are you using when you are running locally?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uplsh580 edited a comment on issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
uplsh580 edited a comment on issue #14249:
URL: https://github.com/apache/airflow/issues/14249#issuecomment-800036386
I am going through the same issue.
In my case
* executor = CeleryExecutor
* sql_alchemy_conn = mysql (version 8.0.23)
* result_backend = mysql (version 8.0.23)
* broker_url = redis (5.0.10)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #14249: both airflow dags test and airflow backfill cli commands got same error in airflow Version 2.0.1
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #14249:
URL: https://github.com/apache/airflow/issues/14249
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org