You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Daniel Halperin (JIRA)" <ji...@apache.org> on 2017/01/26 00:16:26 UTC
[jira] [Comment Edited] (BEAM-1315) FileBasedSink could write tmp
files above the output prefix.
[ https://issues.apache.org/jira/browse/BEAM-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838874#comment-15838874 ]
Daniel Halperin edited comment on BEAM-1315 at 1/26/17 12:15 AM:
-----------------------------------------------------------------
This means that in fact temp file cleanup will cause data loss, or at least failed jobs, if there are multiple sinks writing to :output1/ and :output2/. This is a release blocker.
Is the issue present in 0.4.0?
was (Author: dhalperi@google.com):
This means that in fact temp file cleanup will cause data loss if there are multiple sinks writing to :output1/ and :output2/. This is a release blocker.
Is the issue present in 0.4.0?
> FileBasedSink could write tmp files above the output prefix.
> ------------------------------------------------------------
>
> Key: BEAM-1315
> URL: https://issues.apache.org/jira/browse/BEAM-1315
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
> Reporter: Pei He
> Priority: Blocker
> Fix For: 0.5.0
>
>
> FileBasedWriteOperation.TemporaryDirectoryBuilder tempDirBuilder =
> new FileBasedWriteOperation.TemporaryDirectoryBuilder();
> tempDirBuilder.apply("/home/output/"),
> apply() returns "/home/temp-beam-output-2017-01-25_22-14-49"
> And, it was not under the output prefix "/home/output/".
> Examples of Path resolve:
> Paths.get("/home/output").resolve("temp-beam");
> Paths.get("/home/output/").resolve("temp-beam");
> /home/output/temp-beam
> /home/output/temp-beam
> Paths.get("/home/output").resolve("..").resolve("temp-beam");
> Paths.get("/home/output/").resolve("..").resolve("temp-beam");
> /home/output/../temp-beam
> /home/output/../temp-beam
> Paths.get("/home/output").resolveSibling("temp-beam");
> Paths.get("/home/output/").resolveSibling("temp-beam");
> /home/temp-beam
> /home/temp-beam
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)