You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/10/07 18:14:20 UTC
[jira] [Commented] (FLINK-4777) ContinuousFileMonitoringFunction
may throw IOException when files are moved
[ https://issues.apache.org/jira/browse/FLINK-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15555822#comment-15555822 ]
ASF GitHub Bot commented on FLINK-4777:
---------------------------------------
GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/2610
[FLINK-4777] catch IOException in ContinuousFileMonitoringFunction
FileSystem.listStatus(path) may throw an IOException when it lists files
and then retrieves their file status. This is quite common, e.g. editors
which create temporary files and move them. The
ContinuousFileMonitoringFunction can only apply a file path filter
afterwards.
The solution is to defer file checks until no exception is caught anymore.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink release-1.1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2610.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2610
----
commit 7cb492410ab083dcc56515a0c653936fe32c2c14
Author: Maximilian Michels <mx...@apache.org>
Date: 2016-10-07T18:06:18Z
[FLINK-4777] catch IOException in ContinuousFileMonitoringFunction
FileSystem.listStatus(path) may throw an IOException when it lists files
and then retrieves their file status. This is quite common, e.g. editors
which create temporary files and move them. The
ContinuousFileMonitoringFunction can only apply a file path filter
afterwards.
The solution is to defer file checks until no exception is caught anymore.
----
> ContinuousFileMonitoringFunction may throw IOException when files are moved
> ---------------------------------------------------------------------------
>
> Key: FLINK-4777
> URL: https://issues.apache.org/jira/browse/FLINK-4777
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
> Affects Versions: 1.2.0, 1.1.2
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Critical
> Fix For: 1.2.0, 1.1.3
>
>
> FileSystem.listStatus(path) may throw an IOException when it lists files and then retrieves their file status. This is quite common, e.g. editors which create temporary files and move them. The ContinuousFileMonitoringFunction can only apply a file path filter afterwards. Thus, this problem can't easily be avoided and needs to be taken care of by the function itself.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)