[jira] [Updated] (TEZ-2732) DefaultSorter throws ArrayIndex exceptions on 2047 Mb size sort buffers

["Hitesh Shah (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hitesh Shah updated TEZ-2732:
-----------------------------
    Affects Version/s: 0.5.0
                       0.6.0
                       0.7.0
                       0.8.0-alpha

> DefaultSorter throws ArrayIndex exceptions on 2047 Mb size sort buffers
> -----------------------------------------------------------------------
>
>                 Key: TEZ-2732
>                 URL: https://issues.apache.org/jira/browse/TEZ-2732
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.5.0, 0.6.0, 0.7.0
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>             Fix For: 0.7.1
>
>         Attachments: TEZ-2732.1.patch, TEZ-2732.branch-0.6-and-0.5.patch, TEZ-2732.branch-0.7.patch
>
>
> {noformat}
>   kvbuffer.length = 2146435072 (2047 MB)
>   Corner case: bufIndex=2026133899, kvbidx=523629312.
>   distkvi = mod - i + j = 2146435072 - 2026133899 + 523629312 = 643930485
>   newPos = (2026133899 + (max(.., min(643930485/2, 271128624))) (This would overflow)
> {noformat}
> Would be good to restrict the max allowed sort buffer to 1800 instead of 2047. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)