You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Jarek Potiuk (Jira)" <ji...@apache.org> on 2020/02/14 09:57:00 UTC
[jira] [Commented] (AIRFLOW-6807) Kubernetes Executor in example
fails with securityContext unknown parameter
[ https://issues.apache.org/jira/browse/AIRFLOW-6807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036852#comment-17036852 ]
Jarek Potiuk commented on AIRFLOW-6807:
---------------------------------------
Logs from there: [https://travis-ci.org/apache/airflow/jobs/650079876?utm_medium=notification&utm_source=github_status]
> Kubernetes Executor in example fails with securityContext unknown parameter
> ---------------------------------------------------------------------------
>
> Key: AIRFLOW-6807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6807
> Project: Apache Airflow
> Issue Type: Bug
> Components: executor-kubernetes
> Affects Versions: 2.0.0
> Reporter: Jarek Potiuk
> Assignee: Daniel Imberman
> Priority: Major
>
> I have found in the logs of kubernetes tests that there is a suspicious error while initialising Pod not sure if this is an error (maybe that is expected?) but I doubt it:
>
> 2020-02-13T21:50:14.522836923Z stdout F [2020-02-13 21:50:14,521] \{base_executor.py:74} INFO - Adding to queue: ['airflow', 'tasks', 'run', 'example_kubernetes_executor_config', 'start_task', '2020-02-13T21:49:38+00:00', '--local', '--pool', 'default_pool', '-sd', '/root/airflow/dags/repo/airflow/example_dags/example_kubernetes_executor_config.py']
> 2020-02-13T21:50:14.527582315Z stdout F [2020-02-13 21:50:14,522] \{kubernetes_executor.py:784} INFO - Add task ('example_kubernetes_executor_config', 'start_task', datetime.datetime(2020, 2, 13, 21, 49, 38, tzinfo=<TimezoneInfo [UTC, GMT, +00:00:00, STD]>), 1) with command ['airflow', 'tasks', 'run', 'example_kubernetes_executor_config', 'start_task', '2020-02-13T21:49:38+00:00', '--local', '--pool', 'default_pool', '-sd', '/root/airflow/dags/repo/airflow/example_dags/example_kubernetes_executor_config.py'] with executor_config \{'KubernetesExecutor': {'annotations': {'test': 'annotation'}}}
> 2020-02-13T21:50:14.547451069Z stdout F [2020-02-13 21:50:14,541] \{kubernetes_executor.py:784} INFO - Add task ('example_kubernetes_executor_config', 'non_root_task', datetime.datetime(2020, 2, 13, 21, 47, 26, tzinfo=<TimezoneInfo [UTC, GMT, +00:00:00, STD]>), 1) with command ['airflow', 'tasks', 'run', 'example_kubernetes_executor_config', 'non_root_task', '2020-02-13T21:47:26+00:00', '--local', '--pool', 'default_pool', '-sd', '/root/airflow/dags/repo/airflow/example_dags/example_kubernetes_executor_config.py'] with executor_config \{'KubernetesExecutor': {'securityContext': {'runAsUser': 1000}}}
> 2020-02-13T21:50:14.559150283Z stdout F [2020-02-13 21:50:14,547] \{scheduler_job.py:1497} ERROR - Exception when executing execute_helper
> 2020-02-13T21:50:14.559280526Z stdout F Traceback (most recent call last):
> 2020-02-13T21:50:14.55929312Z stdout F File "/opt/airflow/airflow/jobs/scheduler_job.py", line 1495, in _execute
> 2020-02-13T21:50:14.559298751Z stdout F self._execute_helper()
> 2020-02-13T21:50:14.559305375Z stdout F File "/opt/airflow/airflow/jobs/scheduler_job.py", line 1555, in _execute_helper
> 2020-02-13T21:50:14.559310254Z stdout F if not self._validate_and_run_task_instances(simple_dag_bag=simple_dag_bag):
> 2020-02-13T21:50:14.559315681Z stdout F File "/opt/airflow/airflow/jobs/scheduler_job.py", line 1608, in _validate_and_run_task_instances
> 2020-02-13T21:50:14.559320866Z stdout F self.executor.heartbeat()
> 2020-02-13T21:50:14.559324659Z stdout F File "/opt/airflow/airflow/executors/base_executor.py", line 148, in heartbeat
> 2020-02-13T21:50:14.559328534Z stdout F self.trigger_tasks(open_slots)
> 2020-02-13T21:50:14.559332568Z stdout F File "/opt/airflow/airflow/executors/base_executor.py", line 180, in trigger_tasks
> 2020-02-13T21:50:14.559336259Z stdout F executor_config=simple_ti.executor_config)
> 2020-02-13T21:50:14.559358373Z stdout F File "/opt/airflow/airflow/executors/kubernetes_executor.py", line 787, in execute_async
> 2020-02-13T21:50:14.559363675Z stdout F kube_executor_config = PodGenerator.from_obj(executor_config)
> 2020-02-13T21:50:14.559368739Z stdout F File "/opt/airflow/airflow/kubernetes/pod_generator.py", line 313, in from_obj
> 2020-02-13T21:50:14.559372715Z stdout F return PodGenerator(**namespaced).gen_pod()
> *2020-02-13T21:50:14.559459476Z stdout F TypeError: __init__() got an unexpected keyword argument 'securityContext'*
> 2020-02-13T21:50:14.569657127Z stdout F [2020-02-13 21:50:14,567] \{helpers.py:236} INFO - Sending Signals.SIGTERM to GPID 29
> 2020-02-13T21:50:14.666643641Z stderr F Process DagFileProcessor19-Process:
> 2020-02-13T21:50:14.667568079Z stderr F Traceback (most recent call last):
> 2020-02-13T21:50:14.668042093Z stderr F File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
> 2020-02-13T21:50:14.668076817Z stderr F self.run()
> 2020-02-13T21:50:14.668288485Z stderr F File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run
> 2020-02-13T21:50:14.668303694Z stderr F self._target(*self._args, **self._kwargs)
> 2020-02-13T21:50:14.66843888Z stderr F File "/opt/airflow/airflow/jobs/scheduler_job.py", line 149, in _run_file_processor
> 2020-02-13T21:50:14.668455408Z stderr F pickle_dags=pickle_dags
> 2020-02-13T21:50:14.668583932Z stderr F File "/opt/airflow/airflow/utils/session.py", line 61, in wrapper
> 2020-02-13T21:50:14.668595635Z stderr F return func(*args, **kwargs)
> 2020-02-13T21:50:14.668728612Z stderr F File "/opt/airflow/airflow/jobs/scheduler_job.py", line 779, in process_file
--
This message was sent by Atlassian Jira
(v8.3.4#803005)