You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Raúl Cumplido (Jira)" <ji...@apache.org> on 2022/07/28 13:13:00 UTC
[jira] [Commented] (ARROW-17223) [C#] DecimalArray incorrectly appends values greater than MaxValue / 2 and less than MinValue / 2
[ https://issues.apache.org/jira/browse/ARROW-17223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572450#comment-17572450 ]
Raúl Cumplido commented on ARROW-17223:
---------------------------------------
cc ~ [~eerhardt]
> [C#] DecimalArray incorrectly appends values greater than MaxValue / 2 and less than MinValue / 2
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-17223
> URL: https://issues.apache.org/jira/browse/ARROW-17223
> Project: Apache Arrow
> Issue Type: Bug
> Components: C#
> Affects Versions: 4.0.0, 4.0.1, 5.0.0, 6.0.0, 6.0.1, 7.0.0, 8.0.0, 6.0.2, 7.0.1
> Reporter: Alexey Smirnov
> Priority: Critical
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I try to append values into Decimal arrays (Decimal128Array or Decimal256Array), but for values greater than Decimal.MaxValue / 2 and less than Decimal.MinValue / 2, it fails to do it correctly.
> For example for DecimalArray128 I created simple unit test:
> {code:c#}
> [Fact]
> public void AppendMaxMinDecimal()
> {
> // Assert
> var builder = new Decimal128Array.Builder(new Decimal128Type(29, 0));
> var max = Decimal.MaxValue;
> var min = Decimal.MinValue;
>
> // Act
> builder.Append(max);
> builder.Append(min);
> // Assert
> var array = builder.Build();
> Assert.Equal(max, array.GetValue(0));
> Assert.Equal(min, array.GetValue(1));
> }
> {code}
> I assume it to work correctly, but get:
> {color:#de350b}_Assert.Equal() Failure Expected: 79228162514264337593543950335 Actual: -1_{color}
> Looks like the root cause is in _GetBytes_ method of _DecimalUtility_ class:
> {code:c#}
> //...
> Span<byte> bigIntBytes = stackalloc byte[12];
> for (int i = 0; i < 3; i++)
> {
> int bit = decimalBits[i];
> Span<byte> intBytes = stackalloc byte[4];
> if (!BitConverter.TryWriteBytes(intBytes, bit))
> throw new OverflowException($"Could not extract bytes from int {bit}");
> for (int j = 0; j < 4; j++)
> {
> bigIntBytes[4 * i + j] = intBytes[j];
> }
> }
> bigInt = new BigInteger(bigIntBytes);
> //...
> {code}
> according to MSDN: "The binary representation of a {{Decimal}} value is 128-bits consisting of a 96-bit integer number, and a 32-bit set of flags representing things such as the sign and scaling factor used to specify what portion of it is a decimal fraction".
> In 12 bytes BigInteger only 95 bits are used to numbers and 1 bit for a sign.
> So code:
> {code:c#}
> var newBigInt = new BigInteger(Decimal.MaxValue);
> var arr = newBigInt.ToByteArray();
> {code}
> will produce array of 13 bytes long, not 12.
> I tried to change
> _Span<byte> bigIntBytes = stackalloc byte[12];_
> to
> _Span<byte> bigIntBytes = stackalloc byte[13];_
> and this solved the issue.
> PR: https://github.com/apache/arrow/pull/13732
--
This message was sent by Atlassian Jira
(v8.20.10#820010)