You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/14 22:06:08 UTC
(datafusion) branch gh-readonly-queue/main/pr-26248-4e43ad6e412a248a6f5729f005a20cc007eb5594 created (now 9dec77584e)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-26248-4e43ad6e412a248a6f5729f005a20cc007eb5594
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 9dec77584e fix: support RunEndEncoded arguments in MIN/MAX (#26248)
This branch includes the following new commits:
new 9dec77584e fix: support RunEndEncoded arguments in MIN/MAX (#26248)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: fix: support RunEndEncoded arguments in MIN/MAX (#26248)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-26248-4e43ad6e412a248a6f5729f005a20cc007eb5594
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 9dec77584ea39d93aa01bba7f880b318b37d2c24
Author: David Greven <da...@outlook.com>
AuthorDate: Wed Oct 14 20:34:33 2026 +0000
fix: support RunEndEncoded arguments in MIN/MAX (#26248)
## Which issue does this PR close?
- Closes #26247.
## Rationale for this change
`MIN` and `MAX` fail with an internal error on `RunEndEncoded` columns,
while they work on `Dictionary` columns. See #26247 for a reproducer.
## What changes are included in this PR?
`MIN`/`MAX` use a user-defined signature, so `get_min_max_result_type`
alone decides the argument type. It unwraps `Dictionary` to its value
type but passes `RunEndEncoded` through unchanged, so no cast is planned
and the accumulator receives a type it has no implementation for. #24565
added `RunEndEncoded` handling to `coerced_from`, but user-defined
signatures don't go through it.
This PR adds a `RunEndEncoded` arm that recurses into the value type.
With the recursion, `RunEndEncoded<_, Dictionary<K, V>>` coerces to `V`,
the same result type `MAX` gives a plain `Dictionary<K, V>` column.
Unwrapping one level would return the correct value, but type it as
`Dictionary<K, V>`.
The existing `Dictionary` arm unwraps only one level. I left it
unchanged, because a dictionary whose values are themselves encoded
seems rare. I'm happy to make it recursive in this PR if you'd prefer
the two arms to match.
## What is the testing strategy for this PR?
- `test_get_min_max_return_type_coerce_run_end_encoded` in `min_max.rs`
covers `RunEndEncoded<Int32, UInt32>` → `UInt32` and
`RunEndEncoded<Int32, Dictionary<UInt32, Utf8>>` → `Utf8`.
- New cases in `run_end_encoded.slt` go through SQL planning, so they
exercise the UDAF's coercion path rather than calling the accumulator
directly:
- ungrouped `MIN`/`MAX`;
- their result types, via `arrow_typeof`;
- grouped `MIN`/`MAX`;
- `MAX` as a window function;
- `MAX` over `RunEndEncoded<Int32, Dictionary<UInt32, Utf8>>`.
Without the fix, the four `RunEndEncoded<Int32, UInt32>` cases fail with
the internal error from #26247.
## Are there any user-facing changes?
`MIN` and `MAX` now accept `RunEndEncoded` arguments and return the
value type. There are no API changes.
Generated-by: Claude Code (Claude Opus 5.5). I reviewed the change and
can answer for it.
---
datafusion/functions-aggregate/src/min_max.rs | 23 ++++++++++
.../sqllogictest/test_files/run_end_encoded.slt | 53 ++++++++++++++++++++++
2 files changed, 76 insertions(+)
diff --git a/datafusion/functions-aggregate/src/min_max.rs b/datafusion/functions-aggregate/src/min_max.rs
index 0cf2837f5e..e12adad906 100644
--- a/datafusion/functions-aggregate/src/min_max.rs
+++ b/datafusion/functions-aggregate/src/min_max.rs
@@ -59,6 +59,9 @@ fn get_min_max_result_type(input_types: &[DataType]) -> Result<Vec<DataType>> {
// TODO add checker, if the value type is complex data type
Ok(vec![dict_value_type.deref().clone()])
}
+ DataType::RunEndEncoded(_, values) => {
+ get_min_max_result_type(&[values.data_type().clone()])
+ }
// TODO add checker for datatype which min and max supported
// For example, the `Struct` and `Map` type are not supported in the MIN and MAX function
_ => Ok(input_types.to_vec()),
@@ -1507,6 +1510,26 @@ mod tests {
Ok(())
}
+ #[test]
+ fn test_get_min_max_return_type_coerce_run_end_encoded() -> Result<()> {
+ let run_ends = Arc::new(Field::new("run_ends", DataType::Int32, false));
+ let ree = |values: DataType| {
+ DataType::RunEndEncoded(
+ Arc::clone(&run_ends),
+ Arc::new(Field::new("values", values, true)),
+ )
+ };
+
+ let result = get_min_max_result_type(&[ree(DataType::UInt32)])?;
+ assert_eq!(result, vec![DataType::UInt32]);
+
+ let dict =
+ DataType::Dictionary(Box::new(DataType::UInt32), Box::new(DataType::Utf8));
+ let result = get_min_max_result_type(&[ree(dict)])?;
+ assert_eq!(result, vec![DataType::Utf8]);
+ Ok(())
+ }
+
#[test]
fn test_min_max_dictionary() -> Result<()> {
let values = StringArray::from(vec!["b", "c", "a", "🦀", "d"]);
diff --git a/datafusion/sqllogictest/test_files/run_end_encoded.slt b/datafusion/sqllogictest/test_files/run_end_encoded.slt
index cbac587b17..f94e7635a3 100644
--- a/datafusion/sqllogictest/test_files/run_end_encoded.slt
+++ b/datafusion/sqllogictest/test_files/run_end_encoded.slt
@@ -147,3 +147,56 @@ SELECT
) AS windowed_sum
FROM sensor_readings
ORDER BY temperature;
+
+# MIN / MAX over REE columns go through the same planning path as SQL users hit,
+# so the coerced argument type comes from get_min_max_result_type.
+statement ok
+CREATE TABLE ree_readings AS
+SELECT
+ sensor_id,
+ arrow_cast(arrow_cast(temperature, 'UInt32'), 'RunEndEncoded("run_ends": non-null Int32, "values": UInt32)') AS temp_u32
+FROM sensor_readings;
+
+# Ungrouped MIN / MAX over a REE integer column
+query II
+SELECT MIN(temp_u32), MAX(temp_u32) FROM ree_readings;
+----
+20 24
+
+# MIN / MAX return the REE column's value type, as they do for Dictionary
+query TT
+SELECT arrow_typeof(MIN(temp_u32)), arrow_typeof(MAX(temp_u32)) FROM ree_readings;
+----
+UInt32 UInt32
+
+# Grouped MIN / MAX over a REE integer column
+query ?II rowsort
+SELECT sensor_id, MIN(temp_u32), MAX(temp_u32)
+FROM ree_readings
+GROUP BY sensor_id;
+----
+sensor_A 22 24
+sensor_B 20 20
+
+# MAX as a window function over a REE column
+query ?I rowsort
+SELECT sensor_id, MAX(temp_u32) OVER (PARTITION BY sensor_id)
+FROM ree_readings;
+----
+sensor_A 24
+sensor_A 24
+sensor_A 24
+sensor_B 20
+
+# MIN / MAX over a REE column whose values are Dictionary-encoded
+query TT
+SELECT MAX(arrow_cast(arrow_cast(sensor_id, 'Dictionary(UInt32, Utf8)'),
+ 'RunEndEncoded("run_ends": non-null Int32, "values": Dictionary(UInt32, Utf8))')),
+ arrow_typeof(MAX(arrow_cast(arrow_cast(sensor_id, 'Dictionary(UInt32, Utf8)'),
+ 'RunEndEncoded("run_ends": non-null Int32, "values": Dictionary(UInt32, Utf8))')))
+FROM sensor_readings;
+----
+sensor_B Utf8
+
+statement ok
+DROP TABLE ree_readings;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org