[jira] [Commented] (BEAM-434) When examples write output to file it creates many output files instead of one

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369465#comment-15369465 ] 

ASF GitHub Bot commented on BEAM-434:
-------------------------------------

GitHub user amitsela opened a pull request:

    https://github.com/apache/incubator-beam/pull/620

    [BEAM-434] When examples write output to file it creates many output files instead of one

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [ ] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [ ] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/amitsela/incubator-beam BEAM-434

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/620.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #620
    
----
commit 166968e8ab94517e3d67cb4ee8468dcd707ee875
Author: Sela <an...@paypal.com>
Date:   2016-07-10T07:26:44Z

    Use withoutSharding() when writing output to files in examples.

----


> When examples write output to file it creates many output files instead of one
> ------------------------------------------------------------------------------
>
>                 Key: BEAM-434
>                 URL: https://issues.apache.org/jira/browse/BEAM-434
>             Project: Beam
>          Issue Type: Bug
>          Components: examples-java
>            Reporter: Amit Sela
>            Assignee: Amit Sela
>            Priority: Minor
>
> When using `TextIO.Write.to("/path/to/output")` without any restrictions on the number of shards, it might generate many output files (depending on your input), for WordCount for example, you'll get as many output files as unique words in your input.
> Since I think examples are expected to execute in a friendly manner to "see" what it does and not optimize for performance in some way, I suggest to use `withoutSharding()` when writing the example output to an output file.
> Examples I could find that behave this way:
> org.apache.beam.examples.WordCount
> org.apache.beam.examples.complete.TfIdf
> org.apache.beam.examples.cookbook.DeDupExample



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)