[arrow] branch master updated: ARROW-12425: [Rust] Fix new_null_array dictionary creation

[al...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

alamb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
     new 7e3deb5  ARROW-12425: [Rust] Fix new_null_array dictionary creation
7e3deb5 is described below

commit 7e3deb556fb87b4bc86a79053f5e2418ea5e3513
Author: Raphael Taylor-Davies <r....@googlemail.com>
AuthorDate: Sun Apr 18 06:52:52 2021 -0400

    ARROW-12425: [Rust] Fix new_null_array dictionary creation
    
    It is my understanding that an arrow array should always have a backing values array, even if the content is all nulls. new_null_array currently violates this as it doesn't allocate the backing store for DictionaryArrays. This causes the concat kernel, and possibly others, to panic with index violations
    
    Signed-off-by: Raphael Taylor-Davies <r....@googlemail.com>
    
    Closes #10072 from tustvold/null-dictionary-creation
    
    Authored-by: Raphael Taylor-Davies <r....@googlemail.com>
    Signed-off-by: Andrew Lamb <an...@nerdnetworks.org>
---
 rust/arrow/src/array/array.rs | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/rust/arrow/src/array/array.rs b/rust/arrow/src/array/array.rs
index 63d41df..95a3117 100644
--- a/rust/arrow/src/array/array.rs
+++ b/rust/arrow/src/array/array.rs
@@ -421,14 +421,17 @@ pub fn new_null_array(data_type: &DataType, length: usize) -> ArrayRef {
         DataType::Union(_) => {
             unimplemented!("Creating null Union array not yet supported")
         }
-        DataType::Dictionary(_, value) => {
+        DataType::Dictionary(key, value) => {
+            let keys = new_null_array(key, length);
+            let keys = keys.data();
+
             make_array(ArrayData::new(
                 data_type.clone(),
                 length,
                 Some(length),
-                Some(MutableBuffer::new_null(length).into()),
+                keys.null_buffer().cloned(),
                 0,
-                vec![MutableBuffer::new(0).into()], // values are empty
+                keys.buffers().into(),
                 vec![new_empty_array(value.as_ref()).data().clone()],
             ))
         }
@@ -629,5 +632,9 @@ mod tests {
 
         let null_array = new_null_array(array.data_type(), 9);
         assert_eq!(&array, &null_array);
+        assert_eq!(
+            array.data().buffers()[0].len(),
+            null_array.data().buffers()[0].len()
+        );
     }
 }