[GitHub] [arrow] paddyhoran commented on a change in pull request #7004: ARROW-3827: [Rust] Implement UnionArray Updated

[GitBox <gi...@apache.org>]

paddyhoran commented on a change in pull request #7004:
URL: https://github.com/apache/arrow/pull/7004#discussion_r412972235



##########
File path: rust/arrow/src/array/union.rs
##########
@@ -0,0 +1,1172 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Contains the `UnionArray` and `UnionBuilder` types.
+//!
+//! Each slot in a `UnionArray` can have a value chosen from a number of types.  Each of the
+//! possible types are named like the fields of a [`StructArray`](crate::array::StructArray).
+//! A `UnionArray` can have two possible memory layouts, "dense" or "sparse".  For more information
+//! on please see the [specification](https://arrow.apache.org/docs/format/Columnar.html#union-layout).
+//!
+//! Builders are provided for `UnionArray`'s involving primitive types.  `UnionArray`'s of nested
+//! types are also supported but not via `UnionBuilder`, see the tests for examples.
+//!
+//! # Example: Dense Memory Layout
+//!
+//! ```
+//! use arrow::array::UnionBuilder;
+//! use arrow::datatypes::{Float64Type, Int32Type};
+//!
+//! # fn main() -> arrow::error::Result<()> {
+//! let mut builder = UnionBuilder::new_dense(3);
+//! builder.append::<Int32Type>("a", 1).unwrap();
+//! builder.append::<Float64Type>("b", 3.0).unwrap();
+//! builder.append::<Int32Type>("a", 4).unwrap();
+//! let union = builder.build().unwrap();
+//!
+//! assert_eq!(union.type_id(0), 0_i8);
+//! assert_eq!(union.type_id(1), 1_i8);
+//! assert_eq!(union.type_id(2), 0_i8);
+//!
+//! assert_eq!(union.value_offset(0), 0_i32);
+//! assert_eq!(union.value_offset(1), 0_i32);
+//! assert_eq!(union.value_offset(2), 1_i32);
+//!
+//! # Ok(())
+//! # }
+//! ```
+//!
+//! # Example: Sparse Memory Layout
+//! ```
+//! use arrow::array::UnionBuilder;
+//! use arrow::datatypes::{Float64Type, Int32Type};
+//!
+//! # fn main() -> arrow::error::Result<()> {
+//! let mut builder = UnionBuilder::new_sparse(3);
+//! builder.append::<Int32Type>("a", 1).unwrap();
+//! builder.append::<Float64Type>("b", 3.0).unwrap();
+//! builder.append::<Int32Type>("a", 4).unwrap();
+//! let union = builder.build().unwrap();
+//!
+//! assert_eq!(union.type_id(0), 0_i8);
+//! assert_eq!(union.type_id(1), 1_i8);
+//! assert_eq!(union.type_id(2), 0_i8);
+//!
+//! assert_eq!(union.value_offset(0), 0_i32);
+//! assert_eq!(union.value_offset(1), 1_i32);
+//! assert_eq!(union.value_offset(2), 2_i32);
+//!
+//! # Ok(())
+//! # }
+//! ```
+use crate::array::{
+    builder::{builder_to_mutable_buffer, mutable_buffer_to_builder, BufferBuilderTrait},
+    make_array, Array, ArrayData, ArrayDataBuilder, ArrayDataRef, ArrayRef,
+    BooleanBufferBuilder, BufferBuilder, Int32BufferBuilder, Int8BufferBuilder,
+};
+use crate::buffer::{Buffer, MutableBuffer};
+use crate::datatypes::*;
+use crate::error::{ArrowError, Result};
+
+use crate::util::bit_util;
+use core::fmt;
+use std::any::Any;
+use std::collections::HashMap;
+use std::mem::size_of;
+
+/// An Array that can represent slots of varying types
+pub struct UnionArray {
+    data: ArrayDataRef,
+    boxed_fields: Vec<ArrayRef>,
+}
+
+impl UnionArray {
+    /// Creates a new `UnionArray`.
+    ///
+    /// Accepts type ids, child arrays and optionally offsets (for dense unions) to create
+    /// a new `UnionArray`.  This method makes no attempt to validate the data provided by the
+    /// caller and assumes that each of the components are correct and consistent with each other.
+    /// See `try_new` for an alternative that validates the data provided.
+    ///
+    /// # Data Consistency
+    ///
+    /// The `type_ids` `Buffer` should contain `i8` values.  These values should be greater than
+    /// zero and must be less than the number of children provided in `child_arrays`.  These values

Review comment:
       I was just going but [this](https://arrow.apache.org/docs/format/Columnar.html#union-layout).
   
   It's not immediately obvious why you would need that.  I'll have to study the C++ implementation.  I'll open a follow up PR.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org