You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/27 10:48:18 UTC
[GitHub] [airflow] edenGlu opened a new issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
edenGlu opened a new issue #11882:
URL: https://github.com/apache/airflow/issues/11882
Hey,
I use
Apache Airflow version: 1.10.12
Kubernetes version: v1.17.0
executor: KubernetesExecutor
I create a task - SubDagOperator with executor=KubernetesExecutor()
I run this task and it work, but I can't clear the task , not from the UI and not from the command .
the error that I get is:
```
'Pickling an AuthenticationString object is '
TypeError: Pickling an AuthenticationString object is disallowed for security reasons
```any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tylermichael commented on issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
tylermichael commented on issue #11882:
URL: https://github.com/apache/airflow/issues/11882#issuecomment-735893235
I found this error in the logs for my subdag task. I'm not sure if they are linked but I thought I'd put it here anyway.
```
ERROR - can't compare offset-naive and offset-aware datetimes
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/operators/subdag_operator.py", line 102, in execute
executor=self.executor)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dag.py", line 1432, in run
job.run()
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/base_job.py", line 218, in run
self._execute()
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 787, in _execute
session=session)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 703, in _execute_for_run_dates
dag_run = self._get_dag_run(next_run_date, session=session)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/jobs/backfill_job.py", line 322, in _get_dag_run
run.verify_integrity(session=session)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/db.py", line 70, in wrapper
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dagrun.py", line 394, in verify_integrity
if task.start_date > self.execution_date and not self.is_backfill:
TypeError: can't compare offset-naive and offset-aware datetimes
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #11882:
URL: https://github.com/apache/airflow/issues/11882#issuecomment-919992437
This issue is reported against old version of Airflow.
Please check if the issue is still present in latest version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #11882:
URL: https://github.com/apache/airflow/issues/11882
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tylermichael commented on issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
tylermichael commented on issue #11882:
URL: https://github.com/apache/airflow/issues/11882#issuecomment-923431095
We upgraded to the latest version a while ago and migrated to task groups. Haven't had any issues since then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11882:
URL: https://github.com/apache/airflow/issues/11882#issuecomment-717154950
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] tylermichael commented on issue #11882: can't clear task SubDagOperator with executor=KubernetesExecutor()
Posted by GitBox <gi...@apache.org>.
tylermichael commented on issue #11882:
URL: https://github.com/apache/airflow/issues/11882#issuecomment-733864559
We are also seeing this. Here is the full stack trace I get when I hit clear. We are running version 1.10.12
```
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/airflow/.local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/airflow/.local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/base.py", line 69, in inner
return self._run_view(f, *args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_admin/base.py", line 368, in _run_view
return fn(self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/flask_login/utils.py", line 258, in decorated_view
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/utils.py", line 290, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/utils.py", line 337, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/www/views.py", line 1341, in clear
include_upstream=upstream)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dag.py", line 1244, in sub_dag
for t in regex_match + also_include}
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/dag.py", line 1244, in <dictcomp>
for t in regex_match + also_include}
File "/usr/local/lib/python3.6/copy.py", line 161, in deepcopy
y = copier(memo)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/models/baseoperator.py", line 690, in __deepcopy__
setattr(result, k, copy.deepcopy(v, memo))
File "/usr/local/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/local/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/local/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/local/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/local/lib/python3.6/copy.py", line 180, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/local/lib/python3.6/copy.py", line 280, in _reconstruct
state = deepcopy(state, memo)
File "/usr/local/lib/python3.6/copy.py", line 150, in deepcopy
y = copier(x, memo)
File "/usr/local/lib/python3.6/copy.py", line 240, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/local/lib/python3.6/copy.py", line 169, in deepcopy
rv = reductor(4)
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 290, in __reduce__
'Pickling an AuthenticationString object is '
TypeError: Pickling an AuthenticationString object is disallowed for security reasons
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org