You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/15 05:44:22 UTC
(datafusion) 01/01: fix: report a negative array_resize size as a user error (#25179)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25179-98d84121cd864e683d24d6405c0b2e95fd7e41a6
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 133111f43a579260df4b0d4e76e2537a46cadda9
Author: Stefan Wang <1f...@gmail.com>
AuthorDate: Tue Sep 15 05:43:58 2026 +0000
fix: report a negative array_resize size as a user error (#25179)
## Which issue does this PR close?
None; found by sweeping the scalar functions with edge-case arguments.
## Rationale for this change
`array_resize` with a negative size raises an internal error, which
tells the
caller they hit a DataFusion bug and asks them to open a report:
```console
> SELECT array_resize(make_array(1,2,3), -1, 0);
Internal error: array_resize: failed to convert size to usize.
This issue was likely caused by a bug in DataFusion's code. Please help us to
resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues
```
The size is an argument to the query, so a negative value is the
caller's input
rather than a broken internal invariant. Sending someone to the issue
tracker
for their own argument is misleading, and it hides what was actually
wrong.
The behaviour was already meant to be an error. `array_resize.slt` has
expected
this call to fail since before this change, but with a bare `query
error`, so
the internal error satisfied it and went unnoticed.
## What changes are included in this PR?
Reject a negative size with an execution error naming the value, which
matches
how the neighbouring maximum-size check already reports bad input:
```console
> SELECT array_resize(make_array(1,2,3), -1, 0);
Execution error: array_resize: size must not be negative, got -1
```
Valid input is untouched: `0` still returns `[]`, `5` still grows to
`[1, 2, 3, 9, 9]`, and a NULL size still returns NULL.
## Are these changes tested?
Yes. The slt file gains the negative case for both `List` and
`LargeList`, plus
`i64::MIN` for the boundary, each pinned to the new message so a
regression back
to an internal error fails the suite.
```console
$ cargo test -p datafusion-sqllogictest --test sqllogictests -- array
Progress: 53/53 files completed (100%)
$ cargo test -p datafusion-functions-nested --lib resize
test result: ok. 4 passed; 0 failed
```
The pre-existing `query error` case for `-5` still passes, so the
contract that
a negative size fails is unchanged.
## Are there any user-facing changes?
The error for a negative size changes from an internal error to an
execution
error with a clear message. No valid call changes behaviour, and no API
changes.
---------
Signed-off-by: 1fanwang <1f...@gmail.com>
Co-authored-by: Neil Conway <ne...@gmail.com>
Co-authored-by: Jeffrey Vo <je...@gmail.com>
---
datafusion/functions-nested/src/resize.rs | 21 +++++++++++++--------
.../sqllogictest/test_files/array/array_resize.slt | 11 +++++++++++
2 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/datafusion/functions-nested/src/resize.rs b/datafusion/functions-nested/src/resize.rs
index 0332ea7d5d..540db7d7fb 100644
--- a/datafusion/functions-nested/src/resize.rs
+++ b/datafusion/functions-nested/src/resize.rs
@@ -24,14 +24,16 @@ use arrow::array::{
};
use arrow::buffer::OffsetBuffer;
use arrow::datatypes::DataType;
-use arrow::datatypes::{ArrowNativeType, Field};
+use arrow::datatypes::Field;
use arrow::datatypes::{
DataType::{LargeList, List},
FieldRef,
};
use datafusion_common::cast::{as_int64_array, as_large_list_array, as_list_array};
use datafusion_common::utils::ListCoercion;
-use datafusion_common::{Result, ScalarValue, exec_err, internal_datafusion_err};
+use datafusion_common::{
+ Result, ScalarValue, exec_datafusion_err, exec_err, internal_datafusion_err,
+};
use datafusion_expr::{
ArrayFunctionArgument, ArrayFunctionSignature, ColumnarValue, Documentation,
ScalarFunctionArgs, ScalarUDFImpl, Signature, TypeSignature, Volatility,
@@ -186,6 +188,13 @@ fn array_resize_inner(arg: &[ArrayRef]) -> Result<ArrayRef> {
}
}
+fn resize_count(count_array: &Int64Array, idx: usize) -> Result<usize> {
+ let c = count_array.value(idx);
+ usize::try_from(c).map_err(|_| {
+ exec_datafusion_err!("array_resize: size must not be negative, got {c}")
+ })
+}
+
/// array_resize keep the original array and append the default element to the end
fn general_list_resize<O: OffsetSizeTrait + TryInto<i64>>(
array: &GenericListArray<O>,
@@ -206,9 +215,7 @@ fn general_list_resize<O: OffsetSizeTrait + TryInto<i64>>(
if array.is_null(row_index) || count_array.is_null(row_index) {
continue;
}
- let target_count = count_array.value(row_index).to_usize().ok_or_else(|| {
- internal_datafusion_err!("array_resize: failed to convert size to usize")
- })?;
+ let target_count = resize_count(count_array, row_index)?;
output_values_len =
output_values_len.checked_add(target_count).ok_or_else(|| {
internal_datafusion_err!("array_resize: output size overflow")
@@ -324,9 +331,7 @@ where
}
null_builder.append_non_null();
- let count = count_array.value(row_index).to_usize().ok_or_else(|| {
- internal_datafusion_err!("array_resize: failed to convert size to usize")
- })?;
+ let count = resize_count(count_array, row_index)?;
let count = O::usize_as(count);
let start = offset_window[0];
if start + count > offset_window[1] {
diff --git a/datafusion/sqllogictest/test_files/array/array_resize.slt b/datafusion/sqllogictest/test_files/array/array_resize.slt
index 37f8f2c693..88c1479d13 100644
--- a/datafusion/sqllogictest/test_files/array/array_resize.slt
+++ b/datafusion/sqllogictest/test_files/array/array_resize.slt
@@ -193,4 +193,15 @@ NULL
[61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 9, 9, 9, 9, 9]
+# a negative size is the caller's error, so it must not surface as an internal error
+query error DataFusion error: Execution error: array_resize: size must not be negative, got \-1
+select array_resize(make_array(1, 2, 3), -1, 0);
+
+query error DataFusion error: Execution error: array_resize: size must not be negative, got \-9223372036854775808
+select array_resize(make_array(1, 2, 3), -9223372036854775808, 0);
+
+query error DataFusion error: Execution error: array_resize: size must not be negative, got \-1
+select array_resize(arrow_cast(make_array(1, 2, 3), 'LargeList(Int64)'), -1, 0);
+
+
include ./cleanup.slt.part
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org