You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Kostas Kloudas (Jira)" <ji...@apache.org> on 2020/08/24 08:50:00 UTC
[jira] [Closed] (FLINK-8046) ContinuousFileMonitoringFunction
wrongly ignores files with exact same timestamp
[ https://issues.apache.org/jira/browse/FLINK-8046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kostas Kloudas closed FLINK-8046.
---------------------------------
Resolution: Duplicate
> ContinuousFileMonitoringFunction wrongly ignores files with exact same timestamp
> --------------------------------------------------------------------------------
>
> Key: FLINK-8046
> URL: https://issues.apache.org/jira/browse/FLINK-8046
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.3.2
> Reporter: Juan Miguel Cejuela
> Priority: Major
> Labels: pull-request-available, stream
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The current monitoring of files sets the internal variable `globalModificationTime` to filter out files that are "older". However, the current test (to check "older") does
> `boolean shouldIgnore = modificationTime <= globalModificationTime;` (rom `shouldIgnore`)
> The comparison should strictly be SMALLER (NOT smaller or equal). The method documentation also states "This happens if the modification time of the file is _smaller_ than...".
> The equality acceptance for "older", makes some files with same exact timestamp to be ignored. The behavior is also non-deterministic, as the first file to be accepted ("first" being pretty much random) makes the rest of files with same exact timestamp to be ignored.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)