You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Piotr Nowojski (Jira)" <ji...@apache.org> on 2020/01/21 09:24:00 UTC

[jira] [Closed] (FLINK-15549) integer overflow in SpillingResettableMutableObjectIterator

     [ https://issues.apache.org/jira/browse/FLINK-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Piotr Nowojski closed FLINK-15549.
----------------------------------
    Fix Version/s: 1.10.0
                   1.9.2
                   1.8.4
       Resolution: Fixed

Merged to master as 4a31751e868d9f6fb7529dd569d703f79abc78ae
release-1.10 as 6b85cc1038eae87d8e99cf5ca136d70a66ec6a70
release-1.9 as 90ad2e6d98a2646f662a7e04e1792595b6a5d6c1
release-1.8 as 22b5831940f80f75b0b4381621abb917bb959347

> integer overflow in SpillingResettableMutableObjectIterator
> -----------------------------------------------------------
>
>                 Key: FLINK-15549
>                 URL: https://issues.apache.org/jira/browse/FLINK-15549
>             Project: Flink
>          Issue Type: Bug
>          Components: API / DataSet
>    Affects Versions: 1.6.4, 1.7.2, 1.8.3, 1.9.1, 1.10.0
>            Reporter: caojian0613
>            Assignee: caojian0613
>            Priority: Major
>              Labels: overflow, pull-request-available
>             Fix For: 1.8.4, 1.9.2, 1.10.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> The SpillingResettableMutableObjectIterator has a data overflow problem if the number of elements in a single input exceeds Integer.MAX_VALUE.
> The reason is inside the SpillingResettableMutableObjectIterator, it track the total number of elements and the number of elements currently read with two int type fileds (elementCount and currentElementNum), and if the number of elements exceeds Integer.MAX_VALUE, it will overflow.
> If there is an overflow, then in the next iteration, after reset the input , the data will not be read or only part of the data will be read.
> Therefore, we should changing the type of these two fields of SpillingResettableIterator* from int to long, and we also need a pre-check mechanism before such numerical.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)