You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "sdf-jkl (via GitHub)" <gi...@apache.org> on 2026/09/30 18:30:37 UTC

[I] variant_get converts integers to decimals without consistently checking target precision [arrow-rs]

sdf-jkl opened a new issue, #11309:
URL: https://github.com/apache/arrow-rs/issues/11309

   ### Describe the bug
   
   `variant_get` can return out-of-precision decimal values when converting Int32 and Int64 variants. Arrow’s cast kernel rejects the same conversions.
   
   The integer branches call `rescale_decimal`, treating Int32 as a precision-9 decimal and Int64 as precision 18. However, these integers can contain 10 and 19 digits respectively. The rescaler’s fast path relies on the declared source precision and can skip output precision validation.
   
   This is similar to #10795 for float-to-decimal conversions.
   
   ### To Reproduce
   
   ```rust
   use arrow::array::{ArrayRef, AsArray, Int32Array};
   use arrow::compute::cast;
   use arrow::datatypes::{DataType, Decimal32Type, DecimalType, Field};
   use parquet_variant::Variant;
   use parquet_variant_compute::{GetOptions, VariantArrayBuilder, variant_get};
   use std::sync::Arc;
   
   let target = DataType::Decimal32(9, 0);
   
   // Arrow cast rejects the value: it exceeds nine digits.
   let integers: ArrayRef = Arc::new(Int32Array::from(vec![1_000_000_000]));
   assert!(cast(&integers, &target).unwrap().is_null(0));
   
   // variant_get returns an out-of-precision decimal.
   let mut builder = VariantArrayBuilder::new(1);
   builder.append_variant(Variant::Int32(1_000_000_000));
   let input = ArrayRef::from(builder.build());
   
   let options = GetOptions::new()
       .with_as_type(Some(Arc::new(Field::new("r", target, true))));
   let out = variant_get(&input, options).unwrap();
   let out = out.as_primitive::<Decimal32Type>();
   
   assert!(!out.is_null(0));
   assert_eq!(out.value(0), 1_000_000_000);
   assert!(!Decimal32Type::is_valid_decimal_precision(out.value(0), 9));
   ```
   
   The same problem occurs with `Variant::Int64(1_000_000_000_000_000_000)` targeting `Decimal64(18, 0)`. Strict mode also accepts these values instead of returning an error.
   
   ### Expected behavior
   
   Match Arrow’s integer-to-decimal cast: return null on precision overflow in safe mode and an error in strict mode.
   
   ### Additional context
   
   A shared checked integer-to-decimal scalar helper could keep the array cast and Variant conversion consistent, following the string conversion consolidation in #10850 and the float conversion work in #11302.
   
   There is also a related negative-scale discrepancy: casting integer `199` to `Decimal32(3, -1)` produces `190`, while `variant_get` produces `200`. Arrow’s integer cast truncates, whereas the decimal rescaler used by Variant rounds.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] variant_get converts integers to decimals without consistently checking target precision [arrow-rs]

Posted by "sdf-jkl (via GitHub)" <gi...@apache.org>.
sdf-jkl commented on issue #11309:
URL: https://github.com/apache/arrow-rs/issues/11309#issuecomment-5917299379

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org