You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Benchao Li (Jira)" <ji...@apache.org> on 2020/01/14 07:57:00 UTC
[jira] [Commented] (FLINK-15581)
SpillingResettableMutableObjectIterator data overflow
[ https://issues.apache.org/jira/browse/FLINK-15581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014911#comment-17014911 ]
Benchao Li commented on FLINK-15581:
------------------------------------
Hi [~pnowojski], I noticed that [~caojian0613] has already created a issue to track this: https://issues.apache.org/jira/browse/FLINK-15549
> SpillingResettableMutableObjectIterator data overflow
> -----------------------------------------------------
>
> Key: FLINK-15581
> URL: https://issues.apache.org/jira/browse/FLINK-15581
> Project: Flink
> Issue Type: Bug
> Components: API / DataSet
> Affects Versions: 1.6.4, 1.7.2, 1.8.3, 1.9.1, 1.10.0
> Reporter: Piotr Nowojski
> Priority: Minor
>
> As [reported by a user on the mailing list|https://lists.apache.org/thread.html/r1e3c53eaddfd8050c94ee4e521da4fc96a119662937cf801801bde52%40%3Cuser.flink.apache.org%3E]
> {quote}
> SpillingResettableMutableObjectIterator has a data overflow problem if the number of elements in a single input exceeds Integer.MAX_VALUE.
> The reason is inside the SpillingResettableMutableObjectIterator, it track the total number of elements and the number of elements currently read with two int type fileds (elementCount and currentElementNum), and if the number of elements exceeds Integer.MAX_VALUE, it will overflow.
> If there is an overflow, then in the next iteration, after reset the input , the data will not be read or only part of the data will be read.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)