[GitHub] [airflow] uranusjr commented on a change in pull request #21328: Rewrite decorated task mapping

[GitBox <gi...@apache.org>]

uranusjr commented on a change in pull request #21328:
URL: https://github.com/apache/airflow/pull/21328#discussion_r802643852



##########
File path: tests/serialization/test_dag_serialization.py
##########
@@ -1654,6 +1654,59 @@ def test_mapped_operator_xcomarg_serde():
     assert xcom_arg.operator is serialized_dag.task_dict['op1']
 
 
+def test_mapped_decorator_serde():
+    from airflow.decorators import task
+    from airflow.models.xcom_arg import XComArg
+    from airflow.serialization.serialized_objects import _XComRef
+
+    with DAG("test-dag", start_date=datetime(2020, 1, 1)) as dag:
+        op1 = BaseOperator(task_id="op1")
+        xcomarg = XComArg(op1, "my_key")
+
+        @task(retry_delay=30)
+        def x(arg1, arg2, arg3, arg4):
+            print(arg1, arg2, arg3, arg4)
+
+        x.partial("foo", arg3=[1, 2, {"a": "b"}]).map({"a": 1, "b": 2}, arg4=xcomarg)
+
+    original = dag.get_task("x")
+
+    serialized = SerializedBaseOperator._serialize(original)
+    assert serialized == {
+        '_is_dummy': False,
+        '_is_mapped': True,
+        '_task_module': 'airflow.decorators.python',
+        '_task_type': '_PythonDecoratedOperator',
+        'downstream_task_ids': [],
+        'partial_kwargs': {
+            'op_args': ["foo"],
+            'op_kwargs': {'arg3': [1, 2, {"__type": "dict", "__var": {'a': 'b'}}]},
+            'retry_delay': 30,
+        },
+        'mapped_kwargs': {
+            'op_args': [{"__type": "dict", "__var": {'a': 1, 'b': 2}}],
+            'op_kwargs': {'arg4': {'__type': 'xcomref', '__var': {'task_id': 'op1', 'key': 'my_key'}}},
+        },
+        'task_id': 'x',
+        'template_ext': [],
+        'template_fields': ['op_args', 'op_kwargs'],
+    }
+
+    deserialized = SerializedBaseOperator.deserialize_operator(serialized)
+    assert isinstance(deserialized, MappedOperator)
+    assert deserialized.deps is MappedOperator.DEFAULT_DEPS
+
+    assert deserialized.mapped_kwargs == {
+        "op_args": [{"a": 1, "b": 2}],
+        "op_kwargs": {"arg4": _XComRef("op1", "my_key")},
+    }
+    assert deserialized.partial_kwargs == {
+        "retry_delay": 30,
+        "op_args": ["foo"],
+        "op_kwargs": {"arg3": [1, 2, {"a": "b"}]},
+    }
+

Review comment:
       At this point `retry_delay` is still `30` verbatim; it become a timedelta only after unmapped. I’ll add a test elsewhere for this.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org