Re: [PR] Guard XCom/DagRun pickle-removal migrations (0049, 0055) against NaN/Infinity and U+0000 in bytea->JSONB conversion [airflow]

["uranusjr (via GitHub)" <gi...@apache.org>]

uranusjr commented on code in PR #69064:
URL: https://github.com/apache/airflow/pull/69064#discussion_r3484424273


##########
airflow-core/src/airflow/migrations/versions/0055_3_0_0_remove_pickled_data_from_dagrun_table.py:
##########
@@ -44,6 +45,28 @@
 airflow_version = "3.0.0"
 
 
+def _json_safe(obj):
+    """Replace non-finite floats with their quoted string form before json.dumps.
+
+    Pickled ``conf`` may contain ``float('nan')`` / ``inf`` / ``-inf``. ``json.dumps``
+    (with the default ``allow_nan=True``) emits the bare tokens ``NaN`` / ``Infinity`` /
+    ``-Infinity``, which PostgreSQL JSON/JSONB rejects. Quoting them to strings mirrors
+    the SQL sanitization in migration 0049 (xcom) so the value is preserved rather than
+    dropped by the per-row error handler below.
+    """
+    if isinstance(obj, float):
+        if math.isnan(obj):
+            return "NaN"
+        if math.isinf(obj):
+            return "Infinity" if obj > 0 else "-Infinity"
+        return obj
+    if isinstance(obj, dict):
+        return {k: _json_safe(v) for k, v in obj.items()}
+    if isinstance(obj, (list, tuple)):
+        return [_json_safe(v) for v in obj]

Review Comment:
   Should these use `Mapping` and `Sequence` from collections.abc instead? Or if we add a case for `str` too, the last isinstance check can be against `Iterable`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org