You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Ludovic Claude (JIRA)" <ji...@apache.org> on 2017/03/06 14:06:32 UTC
[jira] [Created] (AIRFLOW-944) Docker operator does not work with
Docker >= 1.19
Ludovic Claude created AIRFLOW-944:
--------------------------------------
Summary: Docker operator does not work with Docker >= 1.19
Key: AIRFLOW-944
URL: https://issues.apache.org/jira/browse/AIRFLOW-944
Project: Apache Airflow
Issue Type: Bug
Components: docker
Affects Versions: Airflow 1.7.1.3, 1.8.0rc4
Environment: Ubuntu 16.04
Reporter: Ludovic Claude
Docker operator does not work when mem_limit is set and Docker version 1.19 or more recent is used.
Here are the logs, I have seen this issue with Airflow 1.7.1.3 and Airflow 1.8.0 rc4.
[2017-03-06 11:37:54,895] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,895] {docker_operator.py:132} INFO - Starting docker container from image hbpmip/mipmap
[2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,902] {models.py:1417} ERROR - mem_limit has been moved to
host_config in API version 1.19
[2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent call last):
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python3.5/dist-packages/airflow/models.py", line 1369, in run
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: result = task_copy.execute(context=context)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/tmp/src/airflow-imaging-plugins/airflow_pipeline/operators/docker_pipeline_operator.py", line 191, in execute
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: logs = super(DockerPipelineOperator, self).execute(context)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python3.5/dist-packages/airflow/operators/docker_operator.py", line 172, in execute
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: user=self.user
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 133, in create_container
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: volume_driver, stop_signal, networking_config,
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py", line 138, in create_container_config
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: return utils.create_container_config(self._version, *args, **kwargs)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: File "/usr/local/lib/python3.5/dist-packages/docker/utils/utils.py", line 1041, in create_container_config
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: 'mem_limit has been moved to host_config in API version 1.19'
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: docker.errors.InvalidVersion: mem_limit has been moved to host_config in API version 1.19
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,903] {models.py:1433} INFO - Marking task as UP_FOR_RETRY
[2017-03-06 11:37:54,912] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,912] {models.py:1462} ERROR - mem_limit has been moved to host_config in API version 1.19
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)