You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by jkff <gi...@git.apache.org> on 2016/10/04 22:51:22 UTC

[GitHub] incubator-beam pull request #1050: Makes FileBasedSink use a temporary direc...

GitHub user jkff opened a pull request:

    https://github.com/apache/incubator-beam/pull/1050

    Makes FileBasedSink use a temporary directory

    When writing to `/path/to/foo`, temporary files would be written to `/path/too/foo-temp-$uid` (or something like that), i.e. as siblings of the final output. That could lead to issues like http://stackoverflow.com/q/39822859/278042
    
    Now, temporary files are written to a path like: `/path/too/temp-beam-foo-$date/$uid`. This way, the temporary files won't match the same glob as the final output files (even though they may still fail to be deleted due to eventual consistency issues).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jkff/incubator-beam file-sink-tmp-dir

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1050.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1050
    
----
commit 1c34acdaf4a0c0697c9646934ac163788133347b
Author: Eugene Kirpichov <ki...@google.com>
Date:   2016-10-04T22:23:27Z

    Makes FileBasedSink use a temporary directory
    
    When writing to /path/to/foo, temporary files would be
    written to /path/too/foo-temp-$uid (or something like that),
    i.e. as siblings of the final output. That could lead
    to issues like http://stackoverflow.com/q/39822859/278042
    
    Now, temporary files are written to a path like:
    /path/too/temp-beam-foo-$date/$uid.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-beam pull request #1050: [BEAM-57] Makes FileBasedSink use a tempo...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-beam/pull/1050


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---