You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by GitBox <gi...@apache.org> on 2022/12/29 13:40:47 UTC
[GitHub] [arrow] 0x26res opened a new issue, #15109: Can't create a non empty StructArray with no field using `StructArray.from_array`
0x26res opened a new issue, #15109:
URL: https://github.com/apache/arrow/issues/15109
### Describe the bug, including details regarding any error messages, version, and platform.
I want to create a StructArray with several rows but no fields/columns:
```
array = pa.StructArray.from_arrays(arrays=[], names=[], mask=pa.array([True, True]))
assert array.type == pa.struct([])
assert len(array) == 0 # This is wrong
```
As commented in the code `len(array)` is wrong and should be 2 (according to the provided mask).
I found a work around but it's not ideal:
```
array = pa.StructArray.from_arrays(
arrays=[pa.nulls(2, pa.null())], names=["DELETE"], mask=pa.array([True, True])
)
assert len(array) == 2
empty_array = array.cast(pa.struct([]))
assert len(empty_array) == 2
assert empty_array.type == pa.struct([])
```
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] 0x26res commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
0x26res commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387386703
Should we change it to:
```
if (children.size() == 0 && null_bitmap == nullptr) {
return Status::Invalid("Can't infer struct array length with 0 child arrays");
}
```
And probably change the code below to infer the size from the `null_bitmap` in that specific case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387425744
That sounds as an option as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15109: Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387347190
We currently have a special case for this in `StructArray.from_arrays` that causes this:
https://github.com/apache/arrow/blob/359f28ba9d62a5e8456d92dfbe5b16b790019edd/python/pyarrow/array.pxi#L2785-L2788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] 0x26res commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
0x26res commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387460124
Hmm, actually in C++ the null_bitmap is in the shape of a buffer: `std::shared_ptr<Buffer> null_bitmap`. Is it possible to infer the size of of the mask from a Buffer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] 0x26res commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
0x26res commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387374201
@jorisvandenbossche should I try to handle the mask argument in that branch of the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387381724
Nope, the C++ method handle this yet:
https://github.com/apache/arrow/blob/359f28ba9d62a5e8456d92dfbe5b16b790019edd/cpp/src/arrow/array/array_nested.cc#L530-L540
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387471941
Ah, no, a Buffer has a size, but this is the size in bytes, which in case of a null bitmap doesn't give you the length of the array.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche closed issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche closed issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
URL: https://github.com/apache/arrow/issues/15109
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on issue #15109: [Python] Can't create a non empty StructArray with no field using `StructArray.from_array`
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #15109:
URL: https://github.com/apache/arrow/issues/15109#issuecomment-1387378851
It might be worth first checking if the C++ code can handle this nowadays, otherwise handling it there sounds good (maybe a `pa.array([[]]*len(mask), pa.struct([]))` might be sufficient)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org