You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "izveigor (via GitHub)" <gi...@apache.org> on 2023/08/04 17:46:26 UTC
[GitHub] [arrow-datafusion] izveigor opened a new issue, #7194: Implement `array_resize` function
izveigor opened a new issue, #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194
### Is your feature request related to a problem or challenge?
## Summary
| Characteristic | Description |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Function name: | `array_resize` |
| Aliases: | `list_resize` |
| Original function?: | No |
| Function Description: | <b>ClickHouse:</b> Changes the length of the array. </br> <b>DuckDB:</b> Resizes the list to contain size elements. Initializes new elements with value or NULL if value is not set. |
| Sources: | [Concept](https://github.com/apache/arrow-datafusion/discussions/6855) [DuckDB](https://duckdb.org/docs/sql/functions/nested) [ClickHouse](https://clickhouse.com/docs/en/sql-reference/functions/array-functions#arrayresize) |
## Examples:
```
SELECT arrayResize([1], 3)
Query id: 2c975fce-0843-485f-9aef-1a8899728cdc
┌─arrayResize([1], 3)─┐
│ [1,0,0] │
└─────────────────────┘
```
```
SELECT arrayResize([[1, 2]], 2)
Query id: 2765406e-c2dd-4208-bf5b-7fe42cd8b682
┌─arrayResize([[1, 2]], 2)─┐
│ [[1,2],[]] │
└──────────────────────────┘
```
```
SELECT arrayResize([1, 2], 4, 5)
Query id: b75e55d1-1abb-4933-b2b1-f9d472308d91
┌─arrayResize([1, 2], 4, 5)─┐
│ [1,2,5,5] │
└───────────────────────────┘
```
```
SELECT arrayResize([[1, 2], [3, 4]], 4, [1, 1])
Query id: ebaeb0e6-b9e1-4a1f-986f-ff546eb37f9d
┌─arrayResize([[1, 2], [3, 4]], 4, [1, 1])─┐
│ [[1,2],[3,4],[1,1],[1,1]] │
└──────────────────────────────────────────┘
```
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] viirya commented on issue #7194: Implement `array_resize` function
Posted by "viirya (via GitHub)" <gi...@apache.org>.
viirya commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1692970779
For the issues no assignment and no others claiming to do, please feel free to pick it. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Implement `array_resize` function [arrow-datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1876719640
> I will work on this.
Ok, I am too busy with work lately...
Sorry for the lack of progress for so long @izveigor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Rachelint commented on issue #7194: Implement `array_resize` function
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1708682082
> Hello, @Rachelint! As I understand, you worry about two questions: "Can `NULL` be an element in array?" and "May I implement DuckDB implementations rather ClickHouse?". I can not give you guarantees about null handling, because that problem has long history and lots of contradictions. If you interest about it, you can read the discussion in [#7142 (comment)](https://github.com/apache/arrow-datafusion/issues/7142#issuecomment-1693517988). Also, I would like to notice @jayzhan211 and his PR: #7338. The answer of the second question is predictably simple: you can use any methods if they have pragmatic value.
>
> I think I can answer all your questions. If you don't comprehend some situations, then you can ask me again :)
Thanks for the reply, it's very helpful to me!I am reading #7142 and #7338 , and I feel it can solve my confusion. I'm really confused about how datafusion intends to handle `NULL`, and thing got worse after I see the different behaviors in `duckdb` and `clickhouse`... I think above discussion can help me a lot !
About `May I implement DuckDB implementations rather ClickHouse`, it is due some behaviors in `clickhouse` confused me actually... And behaviors in `duckdb` seem more understandable.
For example, this query is no problem in `clickhouse`:
```
SELECT arrayResize([1, 2, 3], 4, NULL)
[1,2,3,NULL]
```
But this query may occur error:
```
SELECT arrayResize([[1], [2], [3]], 4, NULL)
Code: 386. DB::Exception: There is no supertype for types Array(UInt8), Nullable(Nothing) because some of them are Array and some of them are not: While processing arrayResize([[1], [2], [3]], 4, NULL). (NO_COMMON_TYPE) (version 23.8.1.2891 (official build))
```
But it may be not so important now, because my main purpose is to know how datafusion plans to handle this, and the discussion may help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Implement `array_resize` function [arrow-datafusion]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1876730114
> > I will work on this.
>
> Ok, I am too busy with work lately... Sorry for the lack of progress for so long @izveigor
I saw there was no progress on this issue, so I tried to pick it up. Your precious discussion is very helpful. Thank you for your contribution! @Rachelint 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Rachelint commented on issue #7194: Implement `array_resize` function
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1692967161
Hi, I'm interested in this issue, can I have a try?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Implement `array_resize` function [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #7194: Implement `array_resize` function
URL: https://github.com/apache/arrow-datafusion/issues/7194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Rachelint commented on issue #7194: Implement `array_resize` function
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1705993850
@izveigor Hi, sorry for reporting no any progress for so long, due to work reasons.
For the `value is not set` case like:
```
SELECT array_resize([[1, 2]], 2)
```
I want to fill `NULL` like `duckdb` rather than `clickhouse`? Is it ok to do that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] Rachelint commented on issue #7194: Implement `array_resize` function
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1693009717
> For the issues no assignment and no others claiming to do, please feel free to pick it. Thanks.
Thanks for advice :rocket:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] izveigor commented on issue #7194: Implement `array_resize` function
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1706812315
Hello, @Rachelint!
As I understand, you worry about two questions: "Can `NULL` be an element in array?" and "May I implement DuckDB implementations rather ClickHouse?".
I can not give you guarantees about null handling, because that problem has long history and lots of contradictions. If you interest about it, you can read the discussion in https://github.com/apache/arrow-datafusion/issues/7142#issuecomment-1693517988. Also, I would like to notice @jayzhan211 and his PR: https://github.com/apache/arrow-datafusion/pull/7338.
The answer of the second question is predictably simple: you can use any methods if they have pragmatic value.
I think I can answer all your questions. If you don't comprehend some situations, then you can ask me again :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Implement `array_resize` function [arrow-datafusion]
Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on issue #7194:
URL: https://github.com/apache/arrow-datafusion/issues/7194#issuecomment-1876703999
I will work on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org