You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/06/03 12:48:27 UTC

[PR] [SPARK-57248][SQL] Skip the dead result null assignment in CheckOverflow codegen when overflow cannot produce null [spark]

LuciferYang opened a new pull request, #56303:
URL: https://github.com/apache/spark/pull/56303

   ### What changes were proposed in this pull request?
   
   This is a sub-task of [SPARK-56908](https://issues.apache.org/jira/browse/SPARK-56908) (reduce generated Java size in whole-stage codegen).
   
   `CheckOverflow.doGenCode` always emitted the result null check after the `toPrecision` call:
   
   ```scala
   nullSafeCodeGen(ctx, ev, eval => {
     s"""
        |${ev.value} = $eval.toPrecision(
        |  ${dataType.precision}, ${dataType.scale}, Decimal.ROUND_HALF_UP(), $nullOnOverflow, $errorContextCode);
        |${ev.isNull} = ${ev.value} == null;
      """.stripMargin
   })
   ```
   
   The result can only become null in the `nullOnOverflow` path: `Decimal.toPrecision` returns null only when `nullOnOverflow` is true and overflow occurs; otherwise it throws on overflow and never returns null. A null child is already handled by `nullSafeCodeGen`, which initializes `ev.isNull` from the child's nullness (`CheckOverflow.nullable` is `true`). So when `nullOnOverflow` is false, `ev.value == null` is always false and the assignment is a dead `ev.isNull = false;` write.
   
   This PR gates the assignment on `nullOnOverflow`:
   
   ```scala
   val setNull = if (nullOnOverflow) s"${ev.isNull} = ${ev.value} == null;" else ""
   ```
   
   `nullOnOverflow` is the right predicate here: `CheckOverflow.nullable` is hardcoded `true`, so gating on `nullable` would be a no-op. The throw-on-overflow path and the error context are unchanged.
   
   ### Why are the changes needed?
   
   To reduce the size of the generated Java in whole-stage codegen, as tracked by SPARK-56908. Under ANSI mode (the default), `nullOnOverflow` is false, so this dead write was emitted on the common path. In current master, decimal arithmetic does its overflow check inline; the standalone `CheckOverflow` node is produced by the decimal encoder/serialization path (e.g. serializing a `BigDecimal` into the internal row at Dataset/case-class/UDF boundaries), which is where this codegen runs.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No. This is a codegen-only change; `eval`, `nullable`, `dataType`, `toString`, and results are unchanged, so SQL output and plan/golden files are unaffected.
   
   ### How was this patch tested?
   
   Existing `DecimalExpressionSuite` "CheckOverflow" coverage plus a new `checkEvaluation` case for the modified template: `nullOnOverflow = false` with no overflow, asserting a non-null result (the path where the now-removed `setNull` lived). `checkEvaluation` runs interpreted and codegen (mutable and unsafe projections).
   
   Note: `DecimalDivideWithOverflowCheck` shares the same `toPrecision(..., nullOnOverflow, ...)` + result-null-check pattern and could get the same treatment as a follow-up; it is intentionally left out of this PR to keep it focused (its codegen has a different shape — hand-rolled `if/else` and `nullable = nullOnOverflow`).
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   Generated-by: Claude Code (Claude Opus 4.8)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57248][SQL] Skip the dead result null assignment in CheckOverflow codegen when overflow cannot produce null [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #56303:
URL: https://github.com/apache/spark/pull/56303#issuecomment-4616989914

   cc @gengliangwang 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57248][SQL] Skip the dead result null assignment in CheckOverflow codegen when overflow cannot produce null [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #56303:
URL: https://github.com/apache/spark/pull/56303#issuecomment-4618451435

   Close this PR after checking against the preceding criteria.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [SPARK-57248][SQL] Skip the dead result null assignment in CheckOverflow codegen when overflow cannot produce null [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang closed pull request #56303: [SPARK-57248][SQL] Skip the dead result null assignment in CheckOverflow codegen when overflow cannot produce null
URL: https://github.com/apache/spark/pull/56303


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org