You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/07/18 04:25:20 UTC

[jira] [Commented] (TRAFODION-1853) Bitwise shifting overflows when extracting long value from byte buffer in JDBC T4 driver.

    [ https://issues.apache.org/jira/browse/TRAFODION-1853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15381694#comment-15381694 ] 

ASF GitHub Bot commented on TRAFODION-1853:
-------------------------------------------

Github user ryzuo closed the pull request at:

    https://github.com/apache/incubator-trafodion/pull/486


> Bitwise shifting overflows when extracting long value from byte buffer in JDBC T4 driver.
> -----------------------------------------------------------------------------------------
>
>                 Key: TRAFODION-1853
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-1853
>             Project: Apache Trafodion
>          Issue Type: Bug
>          Components: client-jdbc-t4
>            Reporter: RuoYu Zuo
>            Assignee: RuoYu Zuo
>            Priority: Critical
>
> There's overflow been found in method extractLong() of class LogicalByteArray, see code below:
>      long extractLong() {
>          long value;
>          if (swap) {
>              value = ((array[loc]) & 0x00000000000000ffL) | ((array[loc + 1] << 8) & 0x000000000000ff00L)
>                      | ((array[loc + 2] << 16) & 0x0000000000ff0000L) | ((array[loc + 3] << 24) & 0x00000000ff000000L)
>                      | ((array[loc + 4] << 32) & 0x000000ff00000000L) | ((array[loc + 5] << 40) & 0x0000ff0000000000L)
>                      | ((array[loc + 6] << 48) & 0x00ff000000000000L) | ((array[loc + 7] << 56) & 0xff00000000000000L);
>          } else {
>              value = ((array[loc + 7]) & 0x00000000000000ffL) | ((array[loc + 6] << 8) & 0x000000000000ff00L)
>                      | ((array[loc + 5] << 16) & 0x0000000000ff0000L) | ((array[loc + 4] << 24) & 0x00000000ff000000L)
>                      | ((array[loc + 3] << 32) & 0x000000ff00000000L) | ((array[loc + 2] << 40) & 0x0000ff0000000000L)
>                      | ((array[loc + 1] << 48) & 0x00ff000000000000L) | ((array[loc] << 56) & 0xff00000000000000L);
>          }
>          loc += 8;
>          return value;
>      }
> By default, the bitwise shifting is based on 4 byte int, thus << 32 and >> 32 is the maximum bits of shifting allowed, larger than this causes overflows. In this case, the long value extracted by this method will never be correct from 5th byte to 8th byte.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)