You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2019/08/12 15:58:22 UTC
[GitHub] [airflow] mik-laj edited a comment on issue #5489: [AIRFLOW-4843]
Allow orchestration via Docker Swarm (SwarmOperator)
mik-laj edited a comment on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
URL: https://github.com/apache/airflow/pull/5489#issuecomment-520485656
@akki
I tested this operator. I was able to run DAG correctly when I configured the TLS correctly.
```
t1 = DockerSwarmOperator(
api_version='auto',
docker_url='tcp://GCP-IP:2376',
tls_ca_cert="/Users/XXX/.docker/machine/machines/swarm-manager-1/ca.pem",
tls_client_cert="/Users/XXX/.docker/machine/certs/cert.pem",
tls_client_key="/Users/XXX/.docker/machine/certs/key.pem",
command='/bin/sleep 10',
image='centos:latest',
auto_remove=True,
task_id='sleep_with_swarm',
)
```
```
[2019-08-12 17:50:39,941] {__init__.py:51} INFO - Using executor SequentialExecutor
[2019-08-12 17:50:40,500] {dagbag.py:86} INFO - Filling up the DagBag from /Users/kamilbregula/airflow/dags
[2019-08-12 17:50:45,949] {taskinstance.py:614} INFO - Dependencies all met for <TaskInstance: docker_swarm_sample.sleep_with_swarm 2018-01-01T00:00:00+00:00 [None]>
[2019-08-12 17:50:45,956] {taskinstance.py:614} INFO - Dependencies all met for <TaskInstance: docker_swarm_sample.sleep_with_swarm 2018-01-01T00:00:00+00:00 [None]>
[2019-08-12 17:50:45,956] {taskinstance.py:832} INFO -
--------------------------------------------------------------------------------
[2019-08-12 17:50:45,957] {taskinstance.py:833} INFO - Starting attempt 1 of 1
[2019-08-12 17:50:45,957] {taskinstance.py:834} INFO -
--------------------------------------------------------------------------------
[2019-08-12 17:50:45,957] {taskinstance.py:853} INFO - Executing <Task(DockerSwarmOperator): sleep_with_swarm> on 2018-01-01T00:00:00+00:00
[2019-08-12 17:50:51,107] {docker_operator.py:257} INFO - Pulling docker image centos:latest
[2019-08-12 17:50:52,445] {docker_operator.py:261} INFO - Pulling from library/centos
[2019-08-12 17:50:52,752] {docker_operator.py:261} INFO - Pulling fs layer
[2019-08-12 17:50:53,264] {docker_operator.py:261} INFO - Downloading
[2019-08-12 17:50:53,327] {docker_operator.py:261} INFO - Downloading
[2019-08-12 17:50:53,451] {docker_operator.py:261} INFO - Downloading
[2019-08-12 17:50:53,552] {docker_operator.py:261} INFO - Downloading
[2019-08-12 17:50:53,664] {docker_operator.py:261} INFO - Downloading
[2019-08-12 17:50:53,709] {docker_operator.py:261} INFO - Verifying Checksum
[2019-08-12 17:50:53,710] {docker_operator.py:261} INFO - Download complete
[2019-08-12 17:50:53,834] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:53,951] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,051] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,171] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,289] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,410] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,519] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,639] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,786] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:54,915] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,039] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,141] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,273] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,392] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,514] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,624] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,742] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,856] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:55,973] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,091] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,213] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,345] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,495] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,602] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,733] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,837] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:56,951] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,061] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,171] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,275] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,380] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,487] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,607] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,764] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:57,943] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,089] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,379] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,452] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,563] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,684] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:58,998] {docker_operator.py:261} INFO - Extracting
[2019-08-12 17:50:59,090] {docker_operator.py:261} INFO - Pull complete
[2019-08-12 17:50:59,091] {docker_operator.py:261} INFO - Digest: sha256:a799dd8a2ded4a83484bbae769d97655392b3f86533ceb7dd96bbac929809f3c
[2019-08-12 17:50:59,092] {docker_operator.py:261} INFO - Status: Downloaded newer image for centos:latest
[2019-08-12 17:50:59,092] {docker_swarm_operator.py:106} INFO - Starting docker service from image centos:latest
[2019-08-12 17:50:59,159] {docker_swarm_operator.py:123} INFO - Service started: {'ID': '3vji4mflkfsyavqb2spl2vauv'}
[2019-08-12 17:51:14,973] {docker_swarm_operator.py:135} INFO - Service status before exiting: complete
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services