You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "jkosh44 (via GitHub)" <gi...@apache.org> on 2025/02/11 17:59:03 UTC
[I] function: `array_prepend` sometimes doesn't work with nested lists [datafusion]
jkosh44 opened a new issue, #14613:
URL: https://github.com/apache/datafusion/issues/14613
### Describe the bug
When using the function `array_prepend` with nested lists an error is sometimes returned. Interestingly some nested types work.
The following types DO NOT work:
- `LargeList(FixedSizeList(n, t))`
- `LargeList(List(t))`
- `List(LargeList(t))`
- `FixedSizeList(n, LargeList(t))`
The following type DO work:
- `LargeList(LargeList(t))`
- `List(List(t))`
- `List(FixedSizeList(n, t))`
- `FixedSizeList(n1, FixedSizeList(n2, t))`
- `FixedSizeList(n, List(t))`
The following other issues may be related:
- https://github.com/apache/datafusion/issues/8249
- https://github.com/apache/datafusion/issues/8249
### To Reproduce
Broken:
```
> select array_prepend(arrow_cast(make_array(1), 'FixedSizeList(1, Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'LargeList(FixedSizeList(1, Int64))'));
Internal error: could not cast value to arrow_array::array::list_array::GenericListArray<i32>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
> select array_prepend(arrow_cast(make_array(1), 'List(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'LargeList(List(Int64))'));
Internal error: could not cast value to arrow_array::array::list_array::GenericListArray<i32>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
> select array_prepend(arrow_cast(make_array(1), 'LargeList(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'List(LargeList(Int64))'));
Internal error: could not cast value to arrow_array::array::list_array::GenericListArray<i64>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
> select array_prepend(arrow_cast(make_array(1), 'LargeList(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'FixedSizeList(3, LargeList(Int64))'));
Internal error: could not cast value to arrow_array::array::list_array::GenericListArray<i64>.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
```
Works:
```
> select array_prepend(arrow_cast(make_array(1), 'LargeList(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'LargeList(LargeList(Int64))'));
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| array_prepend(arrow_cast(make_array(Int64(1)),Utf8("LargeList(Int64)")),arrow_cast(make_array(make_array(Int64(2)),make_array(Int64(3)),make_array(Int64(4))),Utf8("LargeList(LargeList(Int64))"))) |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [[1], [2], [3], [4]] |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.005 seconds.
> select array_prepend(arrow_cast(make_array(1), 'List(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'List(List(Int64))'));
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| array_prepend(arrow_cast(make_array(Int64(1)),Utf8("List(Int64)")),arrow_cast(make_array(make_array(Int64(2)),make_array(Int64(3)),make_array(Int64(4))),Utf8("List(List(Int64))"))) |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [[1], [2], [3], [4]] |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.004 seconds.
> select array_prepend(arrow_cast(make_array(1), 'FixedSizeList(1, Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'List(FixedSizeList(1, Int64))'));
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| array_prepend(arrow_cast(make_array(Int64(1)),Utf8("FixedSizeList(1, Int64)")),arrow_cast(make_array(make_array(Int64(2)),make_array(Int64(3)),make_array(Int64(4))),Utf8("List(FixedSizeList(1, Int64))"))) |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [[1], [2], [3], [4]] |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.005 seconds.
> select array_prepend(arrow_cast(make_array(1), 'FixedSizeList(1, Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'FixedSizeList(3, FixedSizeList(1, Int64))'));
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| array_prepend(arrow_cast(make_array(Int64(1)),Utf8("FixedSizeList(1, Int64)")),arrow_cast(make_array(make_array(Int64(2)),make_array(Int64(3)),make_array(Int64(4))),Utf8("FixedSizeList(3, FixedSizeList(1, Int64))"))) |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [[1], [2], [3], [4]] |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.004 seconds.
> select array_prepend(arrow_cast(make_array(1), 'List(Int64)'), arrow_cast(make_array(make_array(2), make_array(3), make_array(4)), 'FixedSizeList(3, List(Int64))'));
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| array_prepend(arrow_cast(make_array(Int64(1)),Utf8("List(Int64)")),arrow_cast(make_array(make_array(Int64(2)),make_array(Int64(3)),make_array(Int64(4))),Utf8("FixedSizeList(3, List(Int64))"))) |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| [[1], [2], [3], [4]] |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.004 seconds.
```
### Expected behavior
The expected behavior is that the function always returns `[[1], [2], [3], [4]]`.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] function: `array_prepend` sometimes doesn't work with nested lists [datafusion]
Posted by "jkosh44 (via GitHub)" <gi...@apache.org>.
jkosh44 commented on issue #14613:
URL: https://github.com/apache/datafusion/issues/14613#issuecomment-2659528279
I just noticed a pattern about the errors. In the `array_prepend` function, `FixedSizeList` is coerced to a `List`. So all of the queries that succeed are coerced to the same inner and outer type. All the queries that fail have different inner and outer types.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] function: `array_prepend` sometimes doesn't work with nested lists [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #14613: function: `array_prepend` sometimes doesn't work with nested lists
URL: https://github.com/apache/datafusion/issues/14613
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] function: `array_prepend` sometimes doesn't work with nested lists [datafusion]
Posted by "friendlymatthew (via GitHub)" <gi...@apache.org>.
friendlymatthew commented on issue #14613:
URL: https://github.com/apache/datafusion/issues/14613#issuecomment-2657928639
Take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org