You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/21 00:09:37 UTC

(datafusion) branch gh-readonly-queue/main/pr-25953-7f89d2e2524b2b2b3e9e7fac177dd01fb6d79f82 created (now e1fb216a10)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-25953-7f89d2e2524b2b2b3e9e7fac177dd01fb6d79f82
in repository https://gitbox.apache.org/repos/asf/datafusion.git


      at e1fb216a10 fix: preserve list element field in array_repeat (#25953)

This branch includes the following new commits:

     new e1fb216a10 fix: preserve list element field in array_repeat (#25953)

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org


(datafusion) 01/01: fix: preserve list element field in array_repeat (#25953)

Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-25953-7f89d2e2524b2b2b3e9e7fac177dd01fb6d79f82
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit e1fb216a10cdab89bcd755ca23c9a6ec4c1c4b04
Author: Stefan Wang <1f...@gmail.com>
AuthorDate: Tue Oct 20 21:33:50 2026 +0000

    fix: preserve list element field in array_repeat (#25953)
    
    ## Which issue does this PR close?
    
    - Closes #25950.
    
    ## Rationale for this change
    
    Calling array_repeat on a list column whose element field is
    non-nullable, or not named item (element is common in external schemas),
    returns an Arrow error instead of a result. Plain List(Int64) works.
    LargeList fails the same way.
    
    ## What changes are included in this PR?
    
    When the first argument is a list, array_repeat rebuilt each repeated
    inner list with a fresh nullable item field. The outer list, and
    return_type, declare the inner type as the input type, so Arrow rejected
    the mismatch. The inner list now reuses the input's own element field,
    so the built array matches the planned return type, for example
    List(List(non-null Int64)).
    
    ## What is the testing strategy for this PR?
    
    New cases in array/array_repeat.slt cover non-null and element-named
    fields for List and LargeList, checking both values and arrow_typeof.
    They fail on main with the error below.
    
    End to end with datafusion-cli:
    
    ```sql
    CREATE TABLE t AS SELECT make_array(1, 2) AS a, arrow_cast(make_array(1, 2), 'List(non-null Int64)') AS b, arrow_cast(make_array(1, 2), 'List(Int64, field: ''element'')') AS c, arrow_cast(make_array(1, 2), 'LargeList(non-null Int64)') AS d;
    SELECT array_repeat(b, 2) FROM t;
    SELECT array_repeat(c, 2) FROM t;
    SELECT array_repeat(d, 2) FROM t;
    ```
    
    ```
    $ ./target/debug/datafusion-cli -f repro.sql
    ```
    
    Before (main):
    
    ```
    Arrow error: Invalid argument error: ListArray expected data type List(non-null Int64) got List(Int64) for "item"
    Arrow error: Invalid argument error: ListArray expected data type List(Int64, field: 'element') got List(Int64) for "item"
    Arrow error: Invalid argument error: LargeListArray expected data type LargeList(non-null Int64) got LargeList(Int64) for "item"
    ```
    
    After:
    
    ```
    +----------------------------+
    | array_repeat(t.b,Int64(2)) |
    +----------------------------+
    | [[1, 2], [1, 2]]           |
    +----------------------------+
    +----------------------------+
    | array_repeat(t.c,Int64(2)) |
    +----------------------------+
    | [[1, 2], [1, 2]]           |
    +----------------------------+
    +----------------------------+
    | array_repeat(t.d,Int64(2)) |
    +----------------------------+
    | [[1, 2], [1, 2]]           |
    +----------------------------+
    ```
    
    ## Are there any user-facing changes?
    
    array_repeat now works on lists with non-nullable or custom-named
    element fields. No API changes.
    
    ---------
    
    Signed-off-by: 1fanwang <1f...@gmail.com>
    Co-authored-by: Neil Conway <ne...@gmail.com>
---
 datafusion/functions-nested/src/repeat.rs          |  2 +-
 .../sqllogictest/test_files/array/array_repeat.slt | 31 ++++++++++++++++++++++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/datafusion/functions-nested/src/repeat.rs b/datafusion/functions-nested/src/repeat.rs
index dc99d78036..832b39b7ff 100644
--- a/datafusion/functions-nested/src/repeat.rs
+++ b/datafusion/functions-nested/src/repeat.rs
@@ -298,7 +298,7 @@ fn general_list_repeat<O: OffsetSizeTrait>(
         None,
     )?;
     let inner_list = GenericListArray::<O>::try_new(
-        Arc::new(Field::new_list_field(list_array.value_type().clone(), true)),
+        Arc::clone(list_array.value_field()),
         OffsetBuffer::new(inner_offsets.into()),
         inner_values,
         Some(NullBuffer::new(inner_nulls.finish())),
diff --git a/datafusion/sqllogictest/test_files/array/array_repeat.slt b/datafusion/sqllogictest/test_files/array/array_repeat.slt
index 97e55e7718..0272c305ba 100644
--- a/datafusion/sqllogictest/test_files/array/array_repeat.slt
+++ b/datafusion/sqllogictest/test_files/array/array_repeat.slt
@@ -271,4 +271,35 @@ statement ok
 drop table array_repeat_null_slot_table
 
 
+# array_repeat keeps a non-nullable or custom-named element field
+statement ok
+create table array_repeat_field_table as select
+  arrow_cast(make_array(1, 2), 'List(non-null Int64)') as non_null_list,
+  arrow_cast(make_array(1, 2), 'List(Int64, field: ''element'')') as named_list,
+  arrow_cast(make_array(1, 2), 'LargeList(non-null Int64)') as non_null_large_list,
+  arrow_cast(make_array(1, 2), 'LargeList(Int64, field: ''element'')') as named_large_list;
+
+query ????
+select
+  array_repeat(non_null_list, 2),
+  array_repeat(named_list, 2),
+  array_repeat(non_null_large_list, 2),
+  array_repeat(named_large_list, 2)
+from array_repeat_field_table;
+----
+[[1, 2], [1, 2]] [[1, 2], [1, 2]] [[1, 2], [1, 2]] [[1, 2], [1, 2]]
+
+query TTTT
+select
+  arrow_typeof(array_repeat(non_null_list, 2)),
+  arrow_typeof(array_repeat(named_list, 2)),
+  arrow_typeof(array_repeat(non_null_large_list, 2)),
+  arrow_typeof(array_repeat(named_large_list, 2))
+from array_repeat_field_table;
+----
+List(List(non-null Int64)) List(List(Int64, field: 'element')) LargeList(LargeList(non-null Int64)) LargeList(LargeList(Int64, field: 'element'))
+
+statement ok
+drop table array_repeat_field_table
+
 include ./cleanup.slt.part


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org