[I] cast from integer to Decimal32/64 silently truncates values that do not fit [arrow-rs]

["yongster (via GitHub)" <gi...@apache.org>]

yongster opened a new issue, #10706:
URL: https://github.com/apache/arrow-rs/issues/10706

   ### Describe the bug
   
   ### Describe the bug
   
   `cast` / `cast_with_options` from an integer array to `Decimal32` or `Decimal64`
   can rewrite the value instead of rejecting it.
   
   The conversion first uses `AsPrimitive` (`as`), which wraps when the source
   integer does not fit the decimal native type (`i32` / `i64`). The precision
   check then runs on the already-truncated value. If that wrapped value happens
   to fit the requested precision, it is stored as if it were the original number.
   
   The same input cast to `Decimal128` is correct: `i64 as i128` is lossless, so
   the precision check still sees the original value and returns null / an error.
   
   
   
   
   
   ### To Reproduce
   
   Reproduced on `main` (59.2.0).
   
   ;
   
       let a = Int64Array::from(vec![5_000_000_000i64]);
   
       let d32 = cast_with_options(&a, &DataType::Decimal32(9, 0), &safe).unwrap();
       println!("i64 5e9 -> Decimal32(9,0) safe: {d32:?}");
       // actual:   PrimitiveArray<Decimal32(9, 0)>[705032704]
       // expected: PrimitiveArray<Decimal32(9, 0)>[null]
   
       let d128 = cast_with_options(&a, &DataType::Decimal128(9, 0), &safe).unwrap();
       println!("i64 5e9 -> Decimal128(9,0) safe: {d128:?}");
       // Decimal128 is already correct: [null]
   
       let u = UInt32Array::from(vec![4_000_000_000u32]);
       let u32_unsafe =
           cast_with_options(&u, &DataType::Decimal32(9, 0), &unsafe_opts);
       println!("u32 4e9 -> Decimal32(9,0) unsafe: {u32_unsafe:?}");
       // actual:   Ok(PrimitiveArray<Decimal32(9, 0)>[-294967296])
       // expected: Err(...)
     let umax = UInt64Array::from(vec![u64::MAX]);
       let u64_unsafe =
           cast_with_options(&umax, &DataType::Decimal64(18, 0), &unsafe_opts);
       println!("u64::MAX -> Decimal64(18,0) unsafe: {u64_unsafe:?}");
       // actual:   Ok(PrimitiveArray<Decimal64(18, 0)>[-1])
       // expected: Err(...)
   }
   
   5_000_000_000i64 as i32 is 705032704. That 9-digit value is then accepted by Decimal32(9, 0) (max is 999_999_999).
   
   ### Expected behavior
   
   Integer to decimal should use the original integer for the range / precision check, then convert.
   
   • If the value does not fit the target native type or the requested precision:
     • safe: true → null
     • safe: false → Err, and the message should mention the original value
   • Int64(5_000_000_000) -> Decimal32(9, 0) should match
     Int64(5_000_000_000) -> Decimal128(9, 0)
   • UInt32(4_000_000_000) -> Decimal32(9, 0) must not become a negative
   • UInt64::MAX -> Decimal64(18, 0) must not become -1
   
   ### Additional context
   
   The conversion is in cast_integer_to_decimal (arrow-cast/src/cast/mod.rs). All four arms do:
   
   v.as_()
       .mul_checked(scale_factor) // or div_checked when scale < 0
       .and_then(|v| D::is_valid_decimal_precision(v, precision).then_some(v))
   
   AsPrimitive is a wrapping cast. The later precision check therefore never sees 5000000000, only 705032704.
   
   Existing overflow tests cover   Int64 -> Decimal128/256   via a  large scale factor, not this native-width wrap into Decimal32 / Decimal64.
   
   I ran the reproduction locally and reviewed the result.
   ---


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] cast from integer to Decimal32/64 silently truncates values that do not fit [arrow-rs]

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey closed issue #10706: cast from integer to Decimal32/64 silently truncates values that do not fit
URL: https://github.com/apache/arrow-rs/issues/10706


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org