[jira] [Updated] (SPARK-58782) DSv2 pushdown serializes a null literal as the string `'null'` instead of SQL `NULL`, returning wrong results

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-58782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-58782:
-----------------------------------
    Labels: pull-request-available  (was: )

> DSv2 pushdown serializes a null literal as the string `'null'` instead of SQL `NULL`, returning wrong results
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-58782
>                 URL: https://issues.apache.org/jira/browse/SPARK-58782
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 4.0.0
>         Environment: Environment independent
>            Reporter: Tim Meehan
>            Priority: Major
>              Labels: pull-request-available
>
> Problem When a null literal appears in a pushed-down DSv2 expression (JDBC aggregate/predicate pushdown), it is serialized into the pushed SQL as the 4-character string literal {{'null'}} instead of the SQL keyword {{{}NULL{}}}. The external engine then executes a different predicate/grouping key than intended and silently returns wrong results. Because a connector {{{}Expression{}}}'s {{describe()}} defaults to {{{}toString(){}}}, the same wrong text also appears in {{EXPLAIN}} output.
> Reproduction (H2, self-contained — no external DB):
> scala
>  
> // table null_literal(s TEXT), rows: 'keep', '', NULL; pushDownAggregate=truesql("SELECT NULLIF(s, '') AS g, COUNT(*) FROM h2.test.null_literal GROUP BY g")
>  
> {{NULLIF(s,'')}} rewrites to {{{}If(s='', null, s){}}}, which is pushed as a {{CASE_WHEN}} grouping key. The generated SQL is:
> sql
>  
> SELECT CASE WHEN "S" = '' THEN 'null' ELSE "S" END, COUNT(*)  FROM ... GROUP BY CASE WHEN "S" = '' THEN 'null' ELSE "S" END
> The {{''}} row is bucketed under the string {{"null"}} instead of collapsing into the real {{NULL}} group — 3 result rows instead of 2.
> Root cause {{LiteralValue.toString}} matches on {{dataType}} before checking the value, so a null {{StringType}} value flows into the {{'…'}} string branch and renders as {{{}'null'{}}}. (A null {{BinaryType}} value hits an {{assert}} and throws; numeric/boolean nulls render as an unquoted {{{}null{}}}.)
> Proposed fix In {{{}LiteralValue.toString{}}}, return the bare keyword {{NULL}} for any null value (guard as the first {{{}case{}}}). A null literal is SQL {{NULL}} regardless of type, so this is dialect-agnostic and fixes every consumer (JDBC dialects, {{{}ToStringSQLBuilder{}}}/EXPLAIN) at once.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org