[GitHub] [arrow-rs] HaoYang670 opened a new issue, #1958: `DecimalArray::from_fixed_size_list_array` fails when `offset > 0`

[GitBox <gi...@apache.org>]

HaoYang670 opened a new issue, #1958:
URL: https://github.com/apache/arrow-rs/issues/1958

   **Describe the bug**
   `from_fixed_size_list_array` cannot return a correct result when `array.offset() > 0`
   
   **To Reproduce**
   ```rust
   
       #[test]
       fn test_decimal_array_from_fixed_size_list() {
           let value_data = ArrayData::builder(DataType::UInt8)
               .len(48)
               .add_buffer(Buffer::from_slice_ref(&[12_i128, 34, 56]))
               .build()
               .unwrap();
           
           let null_buffer = Buffer::from_slice_ref(&[0b_101]);
   
           // Construct a list array from the above two
           let list_data_type = DataType::FixedSizeList(
               Box::new(Field::new("item", DataType::UInt8, false)),
               16,
           );
           let list_data = ArrayData::builder(list_data_type.clone())
               .len(2)
               .null_bit_buffer(Some(null_buffer))
               .offset(1)
               .add_child_data(value_data.clone())
               .build()
               .unwrap();
           let list_array = FixedSizeListArray::from(list_data);
           let decimal = DecimalArray::from_fixed_size_list_array(list_array, 38, 0);
   
           assert_eq!(decimal.len(), 2);
           assert!(decimal.is_null(0));
           assert_eq!(decimal.value_as_string(1), "56".to_string());
       }
   ```
   **Expected behavior**
   Pass the test.
   
   **Additional context**
   The reason  of the failure is that we drop the `offset` when copying `value_buffer` and `null_buffer:
   https://github.com/apache/arrow-rs/blob/master/arrow/src/array/array_binary.rs#L846-L849
   
   A straight way is to add the offset:
   ```rust
           let builder = ArrayData::builder(DataType::Decimal(precision, scale))
               .len(v.len())
               .add_buffer(v.data_ref().child_data()[0].buffers()[0].clone())
               .null_bit_buffer(v.data_ref().null_buffer().cloned())
               .offset(v.offset());
   ```
   
   However, the answer is still not 100% correct when the offset of the child data  > 0. (Nested offsets always make me confused 😅).
   
   So that the way I prefer is to build decimal array from `FixedSizeBinaryArray`, and drop this method, so that we can avoid nested offsets. Maybe we can file another issue to track this.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold commented on issue #1958: `DecimalArray::from_fixed_size_list_array` fails when `offset > 0`

[GitBox <gi...@apache.org>]

tustvold commented on issue #1958:
URL: https://github.com/apache/arrow-rs/issues/1958#issuecomment-1169874389

   This is related to https://github.com/apache/arrow-rs/issues/1726


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-rs] tustvold closed issue #1958: `DecimalArray::from_fixed_size_list_array` fails when `offset > 0`

[GitBox <gi...@apache.org>]

tustvold closed issue #1958: `DecimalArray::from_fixed_size_list_array` fails when `offset > 0`
URL: https://github.com/apache/arrow-rs/issues/1958


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org