You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "liamzwbao (via GitHub)" <gi...@apache.org> on 2025/11/07 00:01:09 UTC
[PR] [Doc] Correct `ListArray` documentation [arrow-rs]
liamzwbao opened a new pull request, #8803:
URL: https://github.com/apache/arrow-rs/pull/8803
# Which issue does this PR close?
- Closes #NNN.
# Rationale for this change
# What changes are included in this PR?
Fix the minor errors in the `ListArray` doc. `slice(1, 3)` should get `value(3) = Null` and `value(4) = D`, so the `value(5)` is unused
# Are these changes tested?
Docs only
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2523870256
##########
arrow-array/src/array/list_array.rs:
##########
Review Comment:
Hi @alamb, while taking a look at [`generic_list_builder`](https://github.com/apache/arrow-rs/blob/f8d957287e59b65f23718f0c2eb67f621d21ac43/arrow-array/src/builder/generic_list_builder.rs#L36-L86), I noticed that we shouldn't increment offset for `NULL` as the top level element. so the logic offsets should be `[[0, 3], [3, 3], [3, 3], [3, 4], [4, 6]]` and the values offsets should be `[0, 3, 3, 3, 4, 6]`, is that right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2519005502
##########
arrow-array/src/array/list_array.rs:
##########
@@ -110,11 +110,11 @@ impl OffsetSizeTrait for i64 {
/// ┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ┌───┐ ┌───┐
/// │ [A,B,C] │ │ (0,3) │ │ 1 │ │ 0 │ │ │ 1 │ │ A │ │ 0 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [] │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
+/// │ [] (empty) │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
/// │ NULL │ │ (3,4) │ │ 0 │ │ 3 │ │ │ 1 │ │ C │ │ 2 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ ? │ │ ? │ │ 3 │
+/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2519005502
##########
arrow-array/src/array/list_array.rs:
##########
@@ -110,11 +110,11 @@ impl OffsetSizeTrait for i64 {
/// ┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ┌───┐ ┌───┐
/// │ [A,B,C] │ │ (0,3) │ │ 1 │ │ 0 │ │ │ 1 │ │ A │ │ 0 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [] │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
+/// │ [] (empty) │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
/// │ NULL │ │ (3,4) │ │ 0 │ │ 3 │ │ │ 1 │ │ C │ │ 2 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ ? │ │ ? │ │ 3 │
+/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
Review Comment:
Makes sense. Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2528932867
##########
arrow-array/src/array/list_array.rs:
##########
Review Comment:
I see -- you are saying that we don't have to have an element in the underlying values array for a null list in the parent. I think either is fine, but your change here is probably clearer.
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#issuecomment-3508544690
> Fix the minor errors in the ListArray doc. slice(1, 3) should get value(3) = Null and value(4) = D, so the value(5) is unused
I think the diagram is actually correct -- the nullability of the ListArray is defined by its own null mask (not the null mask of the child)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2511609636
##########
arrow-array/src/array/list_array.rs:
##########
@@ -110,11 +110,11 @@ impl OffsetSizeTrait for i64 {
/// ┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ┌───┐ ┌───┐
/// │ [A,B,C] │ │ (0,3) │ │ 1 │ │ 0 │ │ │ 1 │ │ A │ │ 0 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [] │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
+/// │ [] (empty) │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
/// │ NULL │ │ (3,4) │ │ 0 │ │ 3 │ │ │ 1 │ │ C │ │ 2 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ ? │ │ ? │ │ 3 │
+/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
Review Comment:
This change is to make it consistent with line 152, but probably unnecessary as
> the nullability of the ListArray is defined by its own null mask (not the null mask of the child)
Perhaps changing the one on line 152 as `?` makes more sense?
cc @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2514963527
##########
arrow-array/src/array/list_array.rs:
##########
@@ -110,11 +110,11 @@ impl OffsetSizeTrait for i64 {
/// ┌─────────────┐ ┌───────┐ │ ┌───┐ ┌───┐ ┌───┐ ┌───┐
/// │ [A,B,C] │ │ (0,3) │ │ 1 │ │ 0 │ │ │ 1 │ │ A │ │ 0 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [] │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
+/// │ [] (empty) │ │ (3,3) │ │ 1 │ │ 3 │ │ │ 1 │ │ B │ │ 1 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
/// │ NULL │ │ (3,4) │ │ 0 │ │ 3 │ │ │ 1 │ │ C │ │ 2 │
/// ├─────────────┤ ├───────┤ │ ├───┤ ├───┤ ├───┤ ├───┤
-/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ ? │ │ ? │ │ 3 │
+/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
Review Comment:
Sorry -- I was confused about this change at first and got mixed up with the whole PR.
I actually think it would be better to not change this line and actually change line 152 also have two `?`,) to emphasize the point that the backing validity and values for a NULL element in a ListArray are not used / ignored
The fact that there are two validity masks (the ListArray and the child) I think makes this topic quite trick
##########
arrow-array/src/array/list_array.rs:
##########
@@ -152,8 +152,8 @@ impl OffsetSizeTrait for i64 {
/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
/// └─────────────┘ └───────┘ │ └───┘ ├───┤ ├───┤ ├───┤
/// │ 5 │ │ │ 1 │ │ D │ │ 4 │
-/// │ └───┘ ├───┤ ├───┤
-/// │ │ 0 │ │ ? │ │ 5 │
+/// │ └───┘ ╠═══╣ ╠═══╣
+/// │ ║ 0 ║ ║ ? ║ │ 5 │
Review Comment:
This change makes sense 👍 thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2525291869
##########
arrow-array/src/array/list_array.rs:
##########
Review Comment:
I mean the example in `generic_list_builder` is correct but the example here in `list_array` is wrong. So we need to change the doc here, let me update the PR to make it clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2524421824
##########
arrow-array/src/array/list_array.rs:
##########
Review Comment:
> nd the values offsets should be [0, 3, 3, 3, 4, 6], is that right?
Yes that sounds right
Isn't that what this line says: https://github.com/apache/arrow-rs/blob/f8d957287e59b65f23718f0c2eb67f621d21ac43/arrow-array/src/builder/generic_list_builder.rs#L79
```rust
&[0, 3, 3, 3, 4, 6]
```
Maybe I don't understand your question
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2511600531
##########
arrow-array/src/array/list_array.rs:
##########
@@ -152,8 +152,8 @@ impl OffsetSizeTrait for i64 {
/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
/// └─────────────┘ └───────┘ │ └───┘ ├───┤ ├───┤ ├───┤
/// │ 5 │ │ │ 1 │ │ D │ │ 4 │
-/// │ └───┘ ├───┤ ├───┤
-/// │ │ 0 │ │ ? │ │ 5 │
+/// │ └───┘ ╠═══╣ ╠═══╣
+/// │ ║ 0 ║ ║ ? ║ │ 5 │
Review Comment:
> > Fix the minor errors in the ListArray doc. slice(1, 3) should get value(3) = Null and value(4) = D, so the value(5) is unused
>
> I think the diagram is actually correct -- the nullability of the ListArray is defined by its own null mask (not the null mask of the child)
For slice(1, 3), the last list in the slice is list_array[3], which is [D]. However, the diagram also shows data coming from list_array[4][0] (the first element of [NULL, F]), i.e. a Null at Values[5]. That element is outside the sliced offsets range so I marked it as unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Doc] Correct `ListArray` documentation [arrow-rs]
Posted by "liamzwbao (via GitHub)" <gi...@apache.org>.
liamzwbao commented on code in PR #8803:
URL: https://github.com/apache/arrow-rs/pull/8803#discussion_r2511600531
##########
arrow-array/src/array/list_array.rs:
##########
@@ -152,8 +152,8 @@ impl OffsetSizeTrait for i64 {
/// │ [D] │ │ (4,5) │ │ 1 │ │ 4 │ │ │ 0 │ │ ? │ │ 3 │
/// └─────────────┘ └───────┘ │ └───┘ ├───┤ ├───┤ ├───┤
/// │ 5 │ │ │ 1 │ │ D │ │ 4 │
-/// │ └───┘ ├───┤ ├───┤
-/// │ │ 0 │ │ ? │ │ 5 │
+/// │ └───┘ ╠═══╣ ╠═══╣
+/// │ ║ 0 ║ ║ ? ║ │ 5 │
Review Comment:
For slice(1, 3), the last list in the slice is list_array[3], which is [D]. However, the diagram also shows data coming from list_array[4][0] (the first element of [NULL, F]), i.e. a Null at Values[5]. That element is outside the sliced offsets range so I marked it as unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org