[GitHub] [airflow] akki commented on a change in pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

[GitBox <gi...@apache.org>]

akki commented on a change in pull request #17474:
URL: https://github.com/apache/airflow/pull/17474#discussion_r689036685



##########
File path: tests/providers/docker/operators/test_docker_swarm.py
##########
@@ -66,22 +65,31 @@ def _client_service_logs_effect():
             mem_limit='128m',
             user='unittest',
             task_id='unittest',
-            mounts=[Mount(source='/host/path', target='/container/path', type='bind')],
+            mounts=[types.Mount(source='/host/path', target='/container/path', type='bind')],
             auto_remove=True,
             tty=True,
+            configs=[types.ConfigReference(config_id="dummy_cfg_id", config_name="dummy_cfg_name")],
+            secrets=[types.SecretReference(secret_id="dummy_secret_id", secret_name="dummy_secret_name")],
+            mode=types.ServiceMode(mode="replicated", replicas=3),
+            networks=["dummy_network"],
         )
         operator.execute(None)
 
         types_mock.TaskTemplate.assert_called_once_with(
-            container_spec=mock_obj, restart_policy=mock_obj, resources=mock_obj
+            container_spec=mock_obj,
+            restart_policy=mock_obj,
+            resources=mock_obj,
+            networks=["dummy_network"],
         )
         types_mock.ContainerSpec.assert_called_once_with(
             image='ubuntu:latest',
             command='env',
             user='unittest',
-            mounts=[Mount(source='/host/path', target='/container/path', type='bind')],
+            mounts=[types.Mount(source='/host/path', target='/container/path', type='bind')],
             tty=True,
             env={'UNIT': 'TEST', 'AIRFLOW_TMP_DIR': '/tmp/airflow'},
+            configs=[types.ConfigReference(config_id="dummy_cfg_id", config_name="dummy_cfg_name")],
+            secrets=[types.SecretReference(secret_id="dummy_secret_id", secret_name="dummy_secret_name")],

Review comment:
       Can we also add an assertion for the `mode` config?
   
   Something like `assert cskwargs['mode'] == types.ServiceMode(mode="replicated", replicas=3)` below the assertions for `name` & `labels`.

##########
File path: airflow/providers/docker/operators/docker_swarm.py
##########
@@ -93,13 +93,40 @@ class DockerSwarmOperator(DockerOperator):
         Supported only if the Docker engine is using json-file or journald logging drivers.
         The `tty` parameter should be set to use this with Python applications.
     :type enable_logging: bool
+    :param configs: List of docker configs to be exposed to the containers of the swarm service.
+        The configs are ConfigReference objects as per the docker api
+        [https://docker-py.readthedocs.io/en/stable/services.html#docker.models.services.ServiceCollection.create]_
+    :type configs: List[docker.types.ConfigReference]
+    :param secrets: List of docker secrets to be exposed to the containers of the swarm service.
+        The secrets are SecretReference objects as per the docker create_service api.
+        [https://docker-py.readthedocs.io/en/stable/services.html#docker.models.services.ServiceCollection.create]_
+    :type secrets: List[docker.types.SecretReference]
+    :param mode: Indicate whether a service should be deployed as a replicated or global service,
+        and associated parameters
+    :type mode: docker.types.ServiceMode
+    :param networks: List of network names or IDs or NetworkAttachmentConfig to attach the service to.
+    :type networks: List[Union[str, NetworkAttachmentConfig]]
     """
 
-    def __init__(self, *, image: str, enable_logging: bool = True, **kwargs) -> None:
+    def __init__(
+        self,
+        *,
+        image: str,
+        enable_logging: bool = True,
+        configs: Optional[List[types.ConfigReference]] = None,

Review comment:
       So users would need to import `docker` in their code to use this feature, right?
   
   I was thinking it might be worth adding the usage example in [example_docker_swarm.py](https://github.com/apache/airflow/blob/b10ed95a2aded01eb5580120ab2abbde1bac633b/airflow/providers/docker/example_dags/example_docker_swarm.py) to make it easy for (new) users to adopt this feature.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org