[jira] [Created] (SPARK-59719) Mark CheckOverflow as null-intolerant to match MakeDecimal

["David Mollitor (Jira)" <ji...@apache.org>]

David Mollitor created SPARK-59719:
--------------------------------------

             Summary: Mark CheckOverflow as null-intolerant to match MakeDecimal
                 Key: SPARK-59719
                 URL: https://issues.apache.org/jira/browse/SPARK-59719
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 4.1.0
            Reporter: David Mollitor


h3. Summary

{{CheckOverflow}} does not declare {{nullIntolerant}}, unlike its sibling {{MakeDecimal}}
(and {{UnscaledValue}}, {{Cast}}, and the arithmetic expressions). It already satisfies the null-intolerance contract at runtime -- it extends {{UnaryExpression}} and overrides
{{nullSafeEval}}, so {{UnaryExpression.eval}} returns null whenever the child is null -- so the property is simply undeclared. When the {{NullIntolerant}} marker trait was replaced by a {{def}} (SPARK-50241), the adjacent {{UnscaledValue}} and {{MakeDecimal}} were updated but {{CheckOverflow}} was missed.

h3. Change

Declare it null-intolerant:

{code:scala}
override def nullIntolerant: Boolean = true
{code}

h3. Why

{{nullIntolerant}} is consumed by the optimizer to:
* fold a null-literal input to a null literal ({{NullPropagation}}), and
* infer {{IsNotNull(child)}} constraints and push down {{IsNotNull}} filters
({{QueryPlanConstraints.scanNullIntolerantAttribute}} / {{InferFiltersFromConstraints}}).

{{CheckOverflow}} wraps decimal results throughout the plan (e.g. the decimal serializers in {{SerializerBuildHelper}}), so declaring the property it already honors lets these optimizations see through it, consistent with {{MakeDecimal}}.

The direction the optimizer relies on holds even though {{CheckOverflow}} with
{{nullOnOverflow = true}} can also return null from a non-null input on overflow: the contract only requires "null in => null out", not the converse -- the same situation as {{Cast}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org