You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Pei He (JIRA)" <ji...@apache.org> on 2016/12/01 22:11:58 UTC
[jira] [Closed] (BEAM-894) Using @Teardown to remove temp files
from failed bundles in Write.WriteBundles
[ https://issues.apache.org/jira/browse/BEAM-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pei He closed BEAM-894.
-----------------------
Resolution: Won't Fix
Fix Version/s: Not applicable
> Using @Teardown to remove temp files from failed bundles in Write.WriteBundles
> ------------------------------------------------------------------------------
>
> Key: BEAM-894
> URL: https://issues.apache.org/jira/browse/BEAM-894
> Project: Beam
> Issue Type: New Feature
> Components: sdk-java-core
> Reporter: Pei He
> Assignee: Pei He
> Fix For: Not applicable
>
>
> FileBasedSink lefts temp files behind for failed bundles, and it forces finalize() to depend on pattern match.
> However, pattern matching is not always reliable for eventual consistency file system, such as GCS.
> Given we now have DoFn.TearDown, we can improve FileBasedSink (and in general Write transform) to remove temp files/resources early when DoFn bundles fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)