You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jedcunningham (via GitHub)" <gi...@apache.org> on 2025/09/05 05:29:22 UTC
[PR] Remove python_callable as string from mapped operator in serialized Dag [airflow]
jedcunningham opened a new pull request, #55288:
URL: https://github.com/apache/airflow/pull/55288
Remove string serialization of python_callable in mapped tasks partial_kwargs. This reduces the serialized Dag size.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove python_callable as string from mapped operator in serialized Dag [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #55288:
URL: https://github.com/apache/airflow/pull/55288#discussion_r2324176300
##########
airflow-core/tests/unit/serialization/test_dag_serialization.py:
##########
@@ -3004,6 +3004,27 @@ def operator_extra_links(self):
assert mapped_task.extra_links == sorted({"airflow", "github"})
+def empty_function(*args, **kwargs):
+ """Empty function for testing."""
+
+
+def test_python_callable_in_partial_kwargs():
+ from airflow.providers.standard.operators.python import PythonOperator
+
+ operator = PythonOperator.partial(
+ task_id="task",
+ python_callable=empty_function,
+ ).expand(op_kwargs=[{"x": 1}])
+
+ serialized = SerializedBaseOperator.serialize_mapped_operator(operator)
+ assert "python_callable" not in serialized["partial_kwargs"]
+ assert serialized["partial_kwargs"]["python_callable_name"] == qualname(empty_function)
+
+ deserialized = SerializedBaseOperator.deserialize_operator(serialized)
+ assert "python_callable" not in getattr(deserialized, "partial_kwargs")
Review Comment:
```suggestion
assert "python_callable" not in deserialized.partial_kwargs
```
##########
airflow-core/tests/unit/serialization/test_dag_serialization.py:
##########
@@ -3004,6 +3004,27 @@ def operator_extra_links(self):
assert mapped_task.extra_links == sorted({"airflow", "github"})
+def empty_function(*args, **kwargs):
+ """Empty function for testing."""
+
+
+def test_python_callable_in_partial_kwargs():
+ from airflow.providers.standard.operators.python import PythonOperator
+
+ operator = PythonOperator.partial(
+ task_id="task",
+ python_callable=empty_function,
+ ).expand(op_kwargs=[{"x": 1}])
+
+ serialized = SerializedBaseOperator.serialize_mapped_operator(operator)
+ assert "python_callable" not in serialized["partial_kwargs"]
+ assert serialized["partial_kwargs"]["python_callable_name"] == qualname(empty_function)
+
+ deserialized = SerializedBaseOperator.deserialize_operator(serialized)
+ assert "python_callable" not in getattr(deserialized, "partial_kwargs")
Review Comment:
```suggestion
assert "python_callable" not in deserialized.partial_kwargs
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove python_callable as string from mapped operator in serialized Dag [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham merged PR #55288:
URL: https://github.com/apache/airflow/pull/55288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove python_callable as string from mapped operator in serialized Dag [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #55288:
URL: https://github.com/apache/airflow/pull/55288#discussion_r2324177051
##########
airflow-core/tests/unit/serialization/test_dag_serialization.py:
##########
@@ -3004,6 +3004,27 @@ def operator_extra_links(self):
assert mapped_task.extra_links == sorted({"airflow", "github"})
+def empty_function(*args, **kwargs):
+ """Empty function for testing."""
+
+
+def test_python_callable_in_partial_kwargs():
+ from airflow.providers.standard.operators.python import PythonOperator
+
+ operator = PythonOperator.partial(
+ task_id="task",
+ python_callable=empty_function,
+ ).expand(op_kwargs=[{"x": 1}])
+
+ serialized = SerializedBaseOperator.serialize_mapped_operator(operator)
+ assert "python_callable" not in serialized["partial_kwargs"]
+ assert serialized["partial_kwargs"]["python_callable_name"] == qualname(empty_function)
+
+ deserialized = SerializedBaseOperator.deserialize_operator(serialized)
+ assert "python_callable" not in getattr(deserialized, "partial_kwargs")
+ assert getattr(deserialized, "partial_kwargs")["python_callable_name"] == qualname(empty_function)
Review Comment:
```suggestion
assert "python_callable" not in deserialized.partial_kwargs
assert deserialized.partial_kwargs["python_callable_name"] == qualname(empty_function)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org