You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Weijun-H (via GitHub)" <gi...@apache.org> on 2026/10/14 06:18:54 UTC
[PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Weijun-H opened a new pull request, #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968
## Which issue does this PR close?
Follow-up to #6476. The existing fallback fixed its incorrect results; this change runs those sort keys natively. RANGE frames that search boundaries over nullable nested keys continue to use Spark as in #6477.
## Rationale for this change
Spark applies `NULLS FIRST` or `NULLS LAST` to the root sort key. It compares null elements inside arrays and null fields inside structs according to the sort direction. Native sorting applied the root null choice recursively, so the existing safeguard sent `ASC NULLS LAST` and `DESC NULLS FIRST` on nullable nested keys to Spark.
The default `array_sort` comparator has a related distinction: null elements go last while nulls inside nested values compare below non-null values. It currently uses the JVM codegen dispatcher even though Comet already has Spark's recursive comparator.
## What changes are included in this PR?
- Wrap affected native sort keys as `struct(isnull(key), key)`, using the sort direction's default null placement for nested comparisons. Keep nondeterministic keys on Spark because wrapping would evaluate the key twice.
- Run default `array_sort` through a native function that uses the existing Spark comparator for nested values and puts null elements last. Custom comparators and arrays with nondefault string collations continue through the JVM dispatcher.
- Update the SQL fixture, native plan/result tests, and compatibility/tuning documentation. RANGE boundary-search fallback remains in place.
## How are these changes tested?
- `make core`
- `./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometExpressionSuite nested sort keys keep root"` (Spark 4.1, JDK 17)
- `./mvnw test -Pspark-3.5 -Dtest=none -Dsuites="org.apache.comet.CometExpressionSuite nested sort keys keep root"` (Spark 3.5, JDK 17)
- `./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometExpressionSuite default array_sort orders null elements last and nested nulls first in native"` (Spark 4.1, JDK 17, with JVM codegen dispatcher disabled in the test)
- `./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometArrayExpressionSuite default array_sort with collated strings keeps Spark comparison"` (Spark 4.1, JDK 17, flat and nested collated strings)
- `./mvnw test -Dtest=none -Dsuites="org.apache.comet.CometSqlFileTestSuite sort_nested_null_order"` (Spark 4.1, JDK 17)
- `cargo test -p datafusion-comet-spark-expr default_array_sort_puts_null_elements_last_and_keeps_null_rows`
- `cargo clippy -p datafusion-comet-spark-expr --lib -- -D warnings`
- `cargo fmt --all --check`, `./mvnw spotless:check -DskipTests`, `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4268963649
##########
docs/source/user-guide/latest/tuning/operators.md:
##########
@@ -187,12 +187,15 @@ nested in arrays and structs, so `ORDER BY`, window ordering and range partition
even with `spark.comet.exec.strictFloatingPoint=true`. Only the comparison key is normalized; returned values keep their
original NaN representation and zero sign.
-A key that nests values in an array or struct whose type can hold a null element or field falls back in every mode under
-`ASC NULLS LAST` or `DESC NULLS FIRST`, and in a `RANGE` window frame that has to find a row's peers, because Spark orders
-such a null below every other value and the native sort and window frames would not
-([#6476](https://github.com/apache/datafusion-comet/issues/6476),
-[#6477](https://github.com/apache/datafusion-comet/issues/6477)). The default null orders run natively, and
-`spark.comet.expression.SortOrder.allowIncompatible=true` forces the other null orders onto the native path.
+A key that nests values in an array or struct whose type can hold a null element or field stays native under
+`ASC NULLS LAST` and `DESC NULLS FIRST` too: Comet separates the key's root null placement from its nested ordering
+([#6476](https://github.com/apache/datafusion-comet/issues/6476)). A `RANGE` window frame that has to find a row's
+peers over such a key still falls back to Spark
+([#6477](https://github.com/apache/datafusion-comet/issues/6477)).
`sort_array` sorts array elements rather than ordering rows. It follows Spark's floating-point ordering as well, so it
also stays native with `spark.comet.exec.strictFloatingPoint=true`.
+
+The default `array_sort` comparator also runs natively. It puts null array elements last while comparing nulls
+inside nested arrays and structs below non-null values. A custom `array_sort` comparator continues through the JVM
+codegen dispatcher.
Review Comment:
The PR behavior (and tests) indicate arrays with non-default string collations also stay on the Spark/JVM path, but this section only mentions custom comparators. Please document the non-default collation limitation here as well so users understand when `array_sort` will not run natively.
##########
spark/src/main/scala/org/apache/comet/serde/CometSortOrder.scala:
##########
@@ -46,26 +46,18 @@ import org.apache.comet.serde.QueryPlanSerde.exprToProtoInternal
*/
object CometSortOrder extends CometExpressionSerde[SortOrder] {
- /**
- * Subjects of both the runtime fallback reasons and the generated compatibility docs. Shared so
- * the two cannot describe the policy differently.
- */
- private val nestedNullOrderSort =
- "Sorting with `ASC NULLS LAST` or `DESC NULLS FIRST` on an array or struct that can hold a " +
- "null element or field"
+ private val nondeterministicNestedKeyReason =
+ "A nested sort key with nondefault root null placement must be deterministic"
Review Comment:
This fallback/unsupported reason doesn’t explain *why* determinism is required. Since the implementation wraps the key and (at least conceptually) evaluates it twice (for `isnull(key)` and `key`), the message should mention duplicate evaluation and that nondeterministic expressions could yield inconsistent results.
##########
spark/src/main/scala/org/apache/comet/serde/CometSortOrder.scala:
##########
@@ -86,7 +78,13 @@ object CometSortOrder extends CometExpressionSerde[SortOrder] {
expr: SortOrder,
inputs: Seq[Attribute],
binding: Boolean): Option[ExprOuterClass.Expr] = {
- val childExpr = exprToProtoInternal(expr.child, inputs, binding)
+ val wrapped = if (needsRootNullWrapper(expr)) {
+ CreateNamedStruct(
+ Seq(Literal("root_null"), IsNull(expr.child), Literal("value"), expr.child))
+ } else {
+ expr.child
+ }
Review Comment:
Wrapping the key as `struct(isnull(child), child)` duplicates the child expression, which can force computing the sort key twice per row in the native expression pipeline. Consider introducing a dedicated wrapper node (or equivalent) in the native/proto representation that computes the child once and reuses it for both fields, or add native common-subexpression elimination for identical subtrees in this sort-key path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4277737328
##########
spark/src/main/scala/org/apache/comet/serde/CometSortOrder.scala:
##########
@@ -46,26 +46,18 @@ import org.apache.comet.serde.QueryPlanSerde.exprToProtoInternal
*/
object CometSortOrder extends CometExpressionSerde[SortOrder] {
- /**
- * Subjects of both the runtime fallback reasons and the generated compatibility docs. Shared so
- * the two cannot describe the policy differently.
- */
- private val nestedNullOrderSort =
- "Sorting with `ASC NULLS LAST` or `DESC NULLS FIRST` on an array or struct that can hold a " +
- "null element or field"
+ private val nondeterministicNestedKeyReason =
+ "A nested sort key with nondefault root null placement must be deterministic"
Review Comment:
The reason now explains the double evaluation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4277736015
##########
docs/source/user-guide/latest/tuning/operators.md:
##########
@@ -187,12 +187,15 @@ nested in arrays and structs, so `ORDER BY`, window ordering and range partition
even with `spark.comet.exec.strictFloatingPoint=true`. Only the comparison key is normalized; returned values keep their
original NaN representation and zero sign.
-A key that nests values in an array or struct whose type can hold a null element or field falls back in every mode under
-`ASC NULLS LAST` or `DESC NULLS FIRST`, and in a `RANGE` window frame that has to find a row's peers, because Spark orders
-such a null below every other value and the native sort and window frames would not
-([#6476](https://github.com/apache/datafusion-comet/issues/6476),
-[#6477](https://github.com/apache/datafusion-comet/issues/6477)). The default null orders run natively, and
-`spark.comet.expression.SortOrder.allowIncompatible=true` forces the other null orders onto the native path.
+A key that nests values in an array or struct whose type can hold a null element or field stays native under
+`ASC NULLS LAST` and `DESC NULLS FIRST` too: Comet separates the key's root null placement from its nested ordering
+([#6476](https://github.com/apache/datafusion-comet/issues/6476)). A `RANGE` window frame that has to find a row's
+peers over such a key still falls back to Spark
+([#6477](https://github.com/apache/datafusion-comet/issues/6477)).
`sort_array` sorts array elements rather than ordering rows. It follows Spark's floating-point ordering as well, so it
also stays native with `spark.comet.exec.strictFloatingPoint=true`.
+
+The default `array_sort` comparator also runs natively. It puts null array elements last while comparing nulls
+inside nested arrays and structs below non-null values. A custom `array_sort` comparator continues through the JVM
+codegen dispatcher.
Review Comment:
Added the non-default collation case in 11722657f.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4274439443
##########
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala:
##########
@@ -195,6 +195,44 @@ class CometExpressionSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}
}
+ test("nested sort keys keep root and interior null ordering separate in native") {
Review Comment:
`nested sort keys keep root and interior null ordering separate in native` repeats `sort_nested_null_order.sql`, which already runs `ORDER BY ... ASC NULLS LAST`, a TopK and a `DENSE_RANK` window over nested null keys, and `query` mode already asserts the plan is native. Could it go?
The `default array_sort` test below it (line 221) fits an SQL file with `-- Config: spark.comet.exec.scalaUDF.codegen.enabled=false`, next to `array_sort_comparator.sql`. It only covers `ARRAY<ARRAY<INT>>` today, so a fixture could add string, binary, decimal, date and timestamp elements, struct elements with null fields, empty and all-null arrays, and a `containsNull = false` input such as `array_sort(array(coalesce(i, 0), 1))`. I could not find a test that reaches the `noNulls` branch of `usesDefaultComparator`. The collation test in `CometArrayExpressionSuite` could move the same way with `-- MinSparkVersion: 4.0` and `expect_fallback`.
##########
native/spark-expr/src/array_funcs/sort_array.rs:
##########
@@ -109,6 +109,62 @@ fn constant(value: &ColumnarValue) -> Option<bool> {
}
}
+/// Spark's default `array_sort` comparator. It puts null elements last, but compares nulls
+/// inside arrays and structs below non-null values.
+#[derive(Debug, Hash, Eq, PartialEq)]
+pub struct SparkArraySort {
+ signature: Signature,
+}
+
+impl Default for SparkArraySort {
+ fn default() -> Self {
+ Self {
+ signature: Signature::any(1, Volatility::Immutable),
+ }
+ }
+}
+
+impl ScalarUDFImpl for SparkArraySort {
+ fn name(&self) -> &str {
+ "spark_array_sort"
+ }
+
+ fn signature(&self) -> &Signature {
+ &self.signature
+ }
+
+ fn return_type(&self, arg_types: &[DataType]) -> Result<DataType> {
+ Ok(arg_types[0].clone())
+ }
+
+ fn invoke_with_args(&self, args: ScalarFunctionArgs) -> Result<ColumnarValue> {
+ let [array] = take_function_args(self.name(), &args.args)?;
+ let is_scalar = matches!(array, ColumnarValue::Scalar(_));
+ let rows = if is_scalar { 1 } else { args.number_rows };
+ let array = array.to_array(rows)?;
+ let Some(list) = array.as_list_opt::<i32>() else {
+ return exec_err!("spark_array_sort takes a list, got {}", array.data_type());
+ };
+ let values = list.values();
+ let compare = spark_comparator(values.as_ref(), values.as_ref())?;
+ let result = sort_by(list, true, |a, b| {
+ match (values.is_null(a), values.is_null(b)) {
Review Comment:
The closure already orders null elements itself, but `spark_comparator` tests the same top-level nulls again inside every comparison, and `values.is_null(a)` and `values.is_null(b)` are two further null-buffer lookups per call. `spark_comparator_ignoring_nulls` exists for callers that apply top-level nulls themselves and `nested_comparison.rs` already uses it. Would it work here? Taking `values.nulls().filter(|n| n.null_count() > 0)` once would also let an array without null elements pass `compare` straight to `sort_by`. The speedup is an expectation and I did not benchmark it.
##########
spark/src/main/scala/org/apache/comet/serde/arrays.scala:
##########
@@ -1172,7 +1172,39 @@ object CometArrayForAll extends CometCodegenDispatch[ArrayForAll]
object CometArrayAggregate extends CometCodegenDispatch[ArrayAggregate]
-object CometArraySort extends CometCodegenDispatch[ArraySort]
+object CometArraySort extends CometCodegenDispatch[ArraySort] with CometTypeShim {
Review Comment:
`GenerateDocs` labels every plain `CometCodegenDispatch` as `Codegen dispatch`, so `docs/source/user-guide/latest/expressions.md` keeps `| array_sort | ✅ | Codegen dispatch | |` although the default comparator now runs natively. `CometArrayFilter` has the same shape, a native fast path for one lambda form and the codegen dispatcher for the rest. It extends `CometExpressionSerde` and its row reads `Native` with a note about the general lambda going through the dispatcher. Could `CometArraySort` follow that pattern and the `array_sort` row get a matching note for custom comparators and non-default collations?
##########
spark/src/test/resources/sql-tests/operators/sort_nested_null_order.sql:
##########
@@ -17,13 +17,8 @@
-- Spark orders a null element of an array key, or a null field of a struct key, below every
-- other value, whatever the key's NULLS FIRST or NULLS LAST, which places only a null key itself.
--- Comet's native sort places a nested null by the key's null order instead, so under ASC NULLS
--- LAST and DESC NULLS FIRST a key whose type can hold one falls back to Spark, in a Sort, a TopK
--- or a window order key (https://github.com/apache/datafusion-comet/issues/6476). The default
--- null orders, ASC NULLS FIRST and DESC NULLS LAST, put it where Spark does and stay native.
---
--- The test harness admits incompatible sort orders by default, so turn that off to check the
--- shipped policy.
+-- For nondefault root NULL placement, Comet wraps the key so only root NULLs use that placement;
+-- nested NULLs keep Spark's direction-dependent order in Sort, TopK and window keys.
-- Config: spark.comet.expression.SortOrder.allowIncompatible=false
Review Comment:
After this change `CometSortOrder.getSupportLevel` returns only `Unsupported` or `Compatible`. On main the only `Incompatible` result was the nested null order case, so I could not find any remaining consumer of `spark.comet.expression.SortOrder.allowIncompatible`. If that is right, this `Config` line, the same line and its comment in `nested_float_order_keys_strict.sql`, and the `getExprAllowIncompatConfigKey("SortOrder") -> "false"` settings the PR adds in `CometExpressionSuite` and `CometWindowExecSuite` have no effect. The override in `CometTestBase` and its comment about mixed zero and negative zero are stale for the same reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4277749499
##########
spark/src/main/scala/org/apache/comet/serde/arrays.scala:
##########
@@ -1172,7 +1172,39 @@ object CometArrayForAll extends CometCodegenDispatch[ArrayForAll]
object CometArrayAggregate extends CometCodegenDispatch[ArrayAggregate]
-object CometArraySort extends CometCodegenDispatch[ArraySort]
+object CometArraySort extends CometCodegenDispatch[ArraySort] with CometTypeShim {
Review Comment:
Done in 11722657f. `CometArraySort` now extends `CometExpressionSerde` like `CometArrayFilter`, so `array_sort` is listed as Native, with a note on custom comparators and non-default collations.
##########
spark/src/test/resources/sql-tests/operators/sort_nested_null_order.sql:
##########
@@ -17,13 +17,8 @@
-- Spark orders a null element of an array key, or a null field of a struct key, below every
-- other value, whatever the key's NULLS FIRST or NULLS LAST, which places only a null key itself.
--- Comet's native sort places a nested null by the key's null order instead, so under ASC NULLS
--- LAST and DESC NULLS FIRST a key whose type can hold one falls back to Spark, in a Sort, a TopK
--- or a window order key (https://github.com/apache/datafusion-comet/issues/6476). The default
--- null orders, ASC NULLS FIRST and DESC NULLS LAST, put it where Spark does and stay native.
---
--- The test harness admits incompatible sort orders by default, so turn that off to check the
--- shipped policy.
+-- For nondefault root NULL placement, Comet wraps the key so only root NULLs use that placement;
+-- nested NULLs keep Spark's direction-dependent order in Sort, TopK and window keys.
-- Config: spark.comet.expression.SortOrder.allowIncompatible=false
Review Comment:
Right, nothing reads it anymore. Removed every remaining setting, including both SQL `Config` lines and the `CometTestBase` override.
##########
spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala:
##########
@@ -195,6 +195,44 @@ class CometExpressionSuite extends CometTestBase with AdaptiveSparkPlanHelper {
}
}
+ test("nested sort keys keep root and interior null ordering separate in native") {
Review Comment:
Removed the duplicate test and moved the `array_sort` coverage into SQL fixtures, adding the requested element types, empty and all-null arrays, and a `containsNull = false` input. The collation case is in `array_sort_collation.sql`.
##########
native/spark-expr/src/array_funcs/sort_array.rs:
##########
@@ -109,6 +109,62 @@ fn constant(value: &ColumnarValue) -> Option<bool> {
}
}
+/// Spark's default `array_sort` comparator. It puts null elements last, but compares nulls
+/// inside arrays and structs below non-null values.
+#[derive(Debug, Hash, Eq, PartialEq)]
+pub struct SparkArraySort {
+ signature: Signature,
+}
+
+impl Default for SparkArraySort {
+ fn default() -> Self {
+ Self {
+ signature: Signature::any(1, Volatility::Immutable),
+ }
+ }
+}
+
+impl ScalarUDFImpl for SparkArraySort {
+ fn name(&self) -> &str {
+ "spark_array_sort"
+ }
+
+ fn signature(&self) -> &Signature {
+ &self.signature
+ }
+
+ fn return_type(&self, arg_types: &[DataType]) -> Result<DataType> {
+ Ok(arg_types[0].clone())
+ }
+
+ fn invoke_with_args(&self, args: ScalarFunctionArgs) -> Result<ColumnarValue> {
+ let [array] = take_function_args(self.name(), &args.args)?;
+ let is_scalar = matches!(array, ColumnarValue::Scalar(_));
+ let rows = if is_scalar { 1 } else { args.number_rows };
+ let array = array.to_array(rows)?;
+ let Some(list) = array.as_list_opt::<i32>() else {
+ return exec_err!("spark_array_sort takes a list, got {}", array.data_type());
+ };
+ let values = list.values();
+ let compare = spark_comparator(values.as_ref(), values.as_ref())?;
+ let result = sort_by(list, true, |a, b| {
+ match (values.is_null(a), values.is_null(b)) {
Review Comment:
Switched to `spark_comparator_ignoring_nulls` and read the null buffer once, so arrays without null elements go straight to `sort_by`. Not benchmarked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] fix: preserve Spark nested null ordering in native sort [datafusion-comet]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on code in PR #6968:
URL: https://github.com/apache/datafusion-comet/pull/6968#discussion_r4277738682
##########
spark/src/main/scala/org/apache/comet/serde/CometSortOrder.scala:
##########
@@ -86,7 +78,13 @@ object CometSortOrder extends CometExpressionSerde[SortOrder] {
expr: SortOrder,
inputs: Seq[Attribute],
binding: Boolean): Option[ExprOuterClass.Expr] = {
- val childExpr = exprToProtoInternal(expr.child, inputs, binding)
+ val wrapped = if (needsRootNullWrapper(expr)) {
+ CreateNamedStruct(
+ Seq(Literal("root_null"), IsNull(expr.child), Literal("value"), expr.child))
+ } else {
+ expr.child
+ }
Review Comment:
Agreed, the key can be evaluated twice; the determinism guard keeps results consistent. A dedicated node or CSE is a broader change, so I'd leave it for a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org