You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/08/02 09:54:43 UTC
[GitHub] [arrow-rs] alamb commented on a diff in pull request #4622: expand docs for FixedSizeListArray
alamb commented on code in PR #4622:
URL: https://github.com/apache/arrow-rs/pull/4622#discussion_r1281677527
##########
arrow-array/src/array/fixed_size_list_array.rs:
##########
@@ -26,7 +26,60 @@ use arrow_schema::{ArrowError, DataType, FieldRef};
use std::any::Any;
use std::sync::Arc;
-/// An array of [fixed size arrays](https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout)
+/// An array of [fixed length lists], similar to JSON arrays
+/// (e.g. `["A", "B"]`).
+///
+/// Lists are represented using a `values` child
+/// array where each list has a fixed size of `value_length`.
+///
+/// Use [`FixedSizeListBuilder`](crate::builder::FixedSizeListBuilder) to
+/// construct a [`FixedSizeListArray`].
+///
+/// # Representation
+///
+/// A [`FixedSizeListArray`] can represent a list of values of any other
+/// supported Arrow type. Each element of the `FixedSizeListArray` itself is
+/// a list which may contain NULL and non-null values,
+/// or may itself be NULL.
+///
+/// For example, this `FixedSizeListArray` stores lists of strings:
+///
+/// ```text
+/// ┌─────────────┐
+/// │ [A,B] │
+/// ├─────────────┤
+/// │ NULL │
+/// ├─────────────┤
+/// │ [C,NULL] │
+/// └─────────────┘
+/// ```
+///
+/// The `values` of this `FixedSizeListArray`s are stored in a child
+/// [`StringArray`] where logical null values take up `values_length` slots in the array
+/// as shown in the following diagram. The logical values
+/// are shown on the left, and the actual `FixedSizeListArray` encoding on the right
+///
+/// ```text
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─┐
+/// ┌─────────────┐ │ ┌───┐ ┌───┐ ┌──────┐ │
+/// │ [A,B] │ │ 1 │ │ │ 1 │ │ A │ │ 0
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ NULL │ │ 0 │ │ │ 1 │ │ B │ │ 1
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ NULL │ │ 2
+/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
+/// | │ 0 │ │ NULL │ │ 3
Review Comment:
When an array element is null, the values can be anything (valid). So I suggest using `????` to show that the contents could be something unknown:
```suggestion
/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ ???? │ │ 2
/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
/// | │ 0 │ │ ???? │ │ 3
```
##########
arrow-array/src/array/fixed_size_list_array.rs:
##########
@@ -60,6 +113,9 @@ use std::sync::Arc;
/// assert_eq!( &[3, 4, 5], list1.as_any().downcast_ref::<Int32Array>().unwrap().values());
/// assert_eq!( &[6, 7, 8], list2.as_any().downcast_ref::<Int32Array>().unwrap().values());
/// ```
+///
+/// [`StringArray`]: crate::array::StringArray
+/// An array of [fixed size arrays](https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout)
Review Comment:
I think this link formatting is a little off
```suggestion
/// [fixed size arrays]: https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout
```
##########
arrow-array/src/array/fixed_size_list_array.rs:
##########
@@ -26,7 +26,60 @@ use arrow_schema::{ArrowError, DataType, FieldRef};
use std::any::Any;
use std::sync::Arc;
-/// An array of [fixed size arrays](https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout)
+/// An array of [fixed length lists], similar to JSON arrays
+/// (e.g. `["A", "B"]`).
+///
+/// Lists are represented using a `values` child
+/// array where each list has a fixed size of `value_length`.
+///
+/// Use [`FixedSizeListBuilder`](crate::builder::FixedSizeListBuilder) to
+/// construct a [`FixedSizeListArray`].
+///
+/// # Representation
+///
+/// A [`FixedSizeListArray`] can represent a list of values of any other
+/// supported Arrow type. Each element of the `FixedSizeListArray` itself is
+/// a list which may contain NULL and non-null values,
+/// or may itself be NULL.
+///
+/// For example, this `FixedSizeListArray` stores lists of strings:
+///
+/// ```text
+/// ┌─────────────┐
+/// │ [A,B] │
+/// ├─────────────┤
+/// │ NULL │
+/// ├─────────────┤
+/// │ [C,NULL] │
+/// └─────────────┘
+/// ```
+///
+/// The `values` of this `FixedSizeListArray`s are stored in a child
+/// [`StringArray`] where logical null values take up `values_length` slots in the array
+/// as shown in the following diagram. The logical values
+/// are shown on the left, and the actual `FixedSizeListArray` encoding on the right
+///
+/// ```text
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─┐
+/// ┌─────────────┐ │ ┌───┐ ┌───┐ ┌──────┐ │
+/// │ [A,B] │ │ 1 │ │ │ 1 │ │ A │ │ 0
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ NULL │ │ 0 │ │ │ 1 │ │ B │ │ 1
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ NULL │ │ 2
+/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
+/// | │ 0 │ │ NULL │ │ 3
+/// Logical Values │ Validity ├───┤ ├──────┤ │
+/// (nulls) │ │ 1 │ │ C │ │ 4
+/// │ ├───┤ ├──────┤ │
+/// │ │ 0 │ │ NULL │ │ 5
Review Comment:
Same thing here:
```suggestion
/// │ │ 0 │ │ ??? │ │ 5
```
##########
arrow-array/src/array/fixed_size_list_array.rs:
##########
@@ -26,7 +26,60 @@ use arrow_schema::{ArrowError, DataType, FieldRef};
use std::any::Any;
use std::sync::Arc;
-/// An array of [fixed size arrays](https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout)
+/// An array of [fixed length lists], similar to JSON arrays
+/// (e.g. `["A", "B"]`).
+///
+/// Lists are represented using a `values` child
+/// array where each list has a fixed size of `value_length`.
+///
+/// Use [`FixedSizeListBuilder`](crate::builder::FixedSizeListBuilder) to
+/// construct a [`FixedSizeListArray`].
+///
+/// # Representation
+///
+/// A [`FixedSizeListArray`] can represent a list of values of any other
+/// supported Arrow type. Each element of the `FixedSizeListArray` itself is
+/// a list which may contain NULL and non-null values,
+/// or may itself be NULL.
+///
+/// For example, this `FixedSizeListArray` stores lists of strings:
+///
+/// ```text
+/// ┌─────────────┐
+/// │ [A,B] │
+/// ├─────────────┤
+/// │ NULL │
+/// ├─────────────┤
+/// │ [C,NULL] │
+/// └─────────────┘
+/// ```
+///
+/// The `values` of this `FixedSizeListArray`s are stored in a child
+/// [`StringArray`] where logical null values take up `values_length` slots in the array
+/// as shown in the following diagram. The logical values
+/// are shown on the left, and the actual `FixedSizeListArray` encoding on the right
+///
+/// ```text
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─┐
+/// ┌─────────────┐ │ ┌───┐ ┌───┐ ┌──────┐ │
+/// │ [A,B] │ │ 1 │ │ │ 1 │ │ A │ │ 0
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ NULL │ │ 0 │ │ │ 1 │ │ B │ │ 1
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ NULL │ │ 2
+/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
+/// | │ 0 │ │ NULL │ │ 3
Review Comment:
When an array element is null, the values can be anything (valid). So I suggest using `????` to show that the contents could be something unknown:
```suggestion
/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ ???? │ │ 2
/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
/// | │ 0 │ │ ???? │ │ 3
```
##########
arrow-array/src/array/fixed_size_list_array.rs:
##########
@@ -26,7 +26,60 @@ use arrow_schema::{ArrowError, DataType, FieldRef};
use std::any::Any;
use std::sync::Arc;
-/// An array of [fixed size arrays](https://arrow.apache.org/docs/format/Columnar.html#fixed-size-list-layout)
+/// An array of [fixed length lists], similar to JSON arrays
+/// (e.g. `["A", "B"]`).
+///
+/// Lists are represented using a `values` child
+/// array where each list has a fixed size of `value_length`.
+///
+/// Use [`FixedSizeListBuilder`](crate::builder::FixedSizeListBuilder) to
+/// construct a [`FixedSizeListArray`].
+///
+/// # Representation
+///
+/// A [`FixedSizeListArray`] can represent a list of values of any other
+/// supported Arrow type. Each element of the `FixedSizeListArray` itself is
+/// a list which may contain NULL and non-null values,
+/// or may itself be NULL.
+///
+/// For example, this `FixedSizeListArray` stores lists of strings:
+///
+/// ```text
+/// ┌─────────────┐
+/// │ [A,B] │
+/// ├─────────────┤
+/// │ NULL │
+/// ├─────────────┤
+/// │ [C,NULL] │
+/// └─────────────┘
+/// ```
+///
+/// The `values` of this `FixedSizeListArray`s are stored in a child
+/// [`StringArray`] where logical null values take up `values_length` slots in the array
+/// as shown in the following diagram. The logical values
+/// are shown on the left, and the actual `FixedSizeListArray` encoding on the right
+///
+/// ```text
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐
+/// ┌ ─ ─ ─ ─ ─ ─ ─ ─┐
+/// ┌─────────────┐ │ ┌───┐ ┌───┐ ┌──────┐ │
+/// │ [A,B] │ │ 1 │ │ │ 1 │ │ A │ │ 0
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ NULL │ │ 0 │ │ │ 1 │ │ B │ │ 1
+/// ├─────────────┤ │ ├───┤ ├───┤ ├──────┤ │
+/// │ [C,NULL] │ │ 1 │ │ │ 0 │ │ NULL │ │ 2
+/// └─────────────┘ │ └───┘ ├───┤ ├──────┤ │
+/// | │ 0 │ │ NULL │ │ 3
+/// Logical Values │ Validity ├───┤ ├──────┤ │
+/// (nulls) │ │ 1 │ │ C │ │ 4
+/// │ ├───┤ ├──────┤ │
+/// │ │ 0 │ │ NULL │ │ 5
Review Comment:
Same thing here:
```suggestion
/// │ │ 0 │ │ ??? │ │ 5
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org