Re: [PR] Add FanOutMapper for one-to-many partition fan-out [airflow]

["phanikumv (via GitHub)" <gi...@apache.org>]

phanikumv commented on code in PR #66030:
URL: https://github.com/apache/airflow/pull/66030#discussion_r3350073188


##########
airflow-core/tests/unit/serialization/test_serialized_objects.py:
##########
@@ -1105,6 +1105,72 @@ def test_decode_product_mapper():
     assert core_pm.to_downstream("2024-06-15T10:30:00|2024-06-15T10:30:00") == "2024-06-15T10|2024-06-15"
 
 
+def test_encode_fan_out_mapper():
+    from airflow.sdk import FanOutMapper, StartOfDayMapper, StartOfWeekMapper, WeekWindow
+    from airflow.serialization.encoders import encode_partition_mapper
+
+    partition_mapper = FanOutMapper(
+        upstream_mapper=StartOfWeekMapper(),
+        window=WeekWindow(),
+        downstream_mapper=StartOfDayMapper(),
+    )
+    assert encode_partition_mapper(partition_mapper) == {
+        Encoding.TYPE: "airflow.partition_mappers.temporal.FanOutMapper",
+        Encoding.VAR: {
+            "upstream_mapper": {
+                Encoding.TYPE: "airflow.partition_mappers.temporal.StartOfWeekMapper",
+                Encoding.VAR: {

Review Comment:
   This is actually causing the Serialization CI jobs to fail.
   
   When a temporal mapper is serialized it always writes a `timezone` field too.This expected dict only lists `input_format` and `output_format`, so the real output has one extra key and the `assert ... ==` fails.
   
     Add the `timezone` key to both nested mappers (it's already present in `test_encode_timezone`, e.g. lines 939/951):
   
     ```suggestion
                 "upstream_mapper": {
                     Encoding.TYPE: "airflow.partition_mappers.temporal.StartOfWeekMapper",
                     Encoding.VAR: {
                         "timezone": "UTC",
                         "input_format": "%Y-%m-%dT%H:%M:%S",
                         "output_format": "%Y-%m-%d (W%V)",
                     },
                 },
     ```
   
     The `downstream_mapper` block just below needs the same `"timezone": "UTC",` line added to its `Encoding.VAR`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org