[jira] [Commented] (DERBY-5536) Client's ResultSet#getLong does not range check when converting from a DECIMAL column

["Dag H. Wanvik (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DERBY-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169620#comment-13169620 ] 

Dag H. Wanvik commented on DERBY-5536:
--------------------------------------

The bug also is present for getInt, getShort and getByte sinc ethey rely on getLong. The error lies in am.Decimal#getLong, which contains this loop:

   // compute the integer part only.
   int leftOfDecimalPoint = length * 2 - 1 - scale;
   long integer = 0;
   if (leftOfDecimalPoint > 0) {
       int i = 0;
       for (; i < leftOfDecimalPoint / 2; i++) {
           integer = integer * 10 + signum * ((buffer[offset + i] & 0xF0) >>> 4); // high nybble.
           integer = integer * 10 + signum * (buffer[offset + i] & 0x0F);        // low nybble.
       }
    :

If the scale is high, say, 31, the accumulating value "integer" will wrap around here with no warning.

                
> Client's ResultSet#getLong does not range check when converting from a DECIMAL column
> -------------------------------------------------------------------------------------
>
>                 Key: DERBY-5536
>                 URL: https://issues.apache.org/jira/browse/DERBY-5536
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC, Network Client
>            Reporter: Dag H. Wanvik
>         Attachments: MissingRangeCheck.java
>
>
> Derby's DECIMAL can contain an integer of 31 digits. This can overflow a long. The embedded JDBC driver catches this, but the client does not and can yield a corrupt long, cf enclosed repro.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira