You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "westonpace (via GitHub)" <gi...@apache.org> on 2024/08/24 17:58:53 UTC

[I] Concatenating dictionaries that have nulls in them yields strange results [arrow-rs]

westonpace opened a new issue, #6302:
URL: https://github.com/apache/arrow-rs/issues/6302

   **Describe the bug**
   If I concatenate two dictionaries that have nulls in the dictionary (not the indices) then I sometimes get garbage back.
   
   **To Reproduce**
   ```
       #[test]
       pub fn test_dict_concat_two() {
           let keys1 = UInt16Array::from_iter_values(vec![2, 2]);
           let keys2 = UInt16Array::from_iter_values(vec![1]);
           let dict1 = StringArray::from(vec![Some("a"), Some("b"), None]);
           let dict2 = StringArray::from(vec![Some("c"), Some("d"), None]);
   
           let arr1 = DictionaryArray::new(keys1, Arc::new(dict1));
           let arr2 = DictionaryArray::new(keys2, Arc::new(dict2));
   
           println!("{:?}", arr1);
           println!("{:?}", arr2);
           let combined = arrow_select::concat::concat(&[&arr1, &arr2]).unwrap();
           println!("{:?}", combined);
   
           let expected = StringArray::from(vec![None, None, Some("d")]);
           let actual = arrow_cast::cast(combined.as_ref(), &DataType::LargeUtf8).unwrap();
   
           assert_eq!(actual.as_ref(), &expected);
       }
   ```
   
   **Expected behavior**
   The test in the reproduction passes.
   
   **Additional context**
   The output from the test is:
   
   ```
   DictionaryArray {keys: PrimitiveArray<UInt16>
   [
     2,
     2,
   ] values: StringArray
   [
     "a",
     "b",
     null,
   ]}
   
   DictionaryArray {keys: PrimitiveArray<UInt16>
   [
     1,
   ] values: StringArray
   [
     "c",
     "d",
     null,
   ]}
   
   DictionaryArray {keys: PrimitiveArray<UInt16>
   [
     0,
     0,
     0,
   ] values: StringArray
   [
     "d",
   ]}
   
   thread 'data::tests::test_dict_concat_two' panicked at rust/lance-encoding/src/data.rs:1179:9:
   assertion `left == right` failed
     left: LargeStringArray
   [
     "d",
     "d",
     "d",
   ]
    right: StringArray
   [
     null,
     null,
     "d",
   ]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Concatenating dictionaries that have nulls in them yields strange results [arrow-rs]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #6302: Concatenating dictionaries that have nulls in them yields strange results
URL: https://github.com/apache/arrow-rs/issues/6302


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org