[jira] [Resolved] (CRUNCH-241) Write side outputs from the Mapper stage of a MapReduce job

["Josh Wills (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CRUNCH-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Wills resolved CRUNCH-241.
-------------------------------

       Resolution: Fixed
    Fix Version/s: 0.7.0

Committed this to master.
                
> Write side outputs from the Mapper stage of a MapReduce job
> -----------------------------------------------------------
>
>                 Key: CRUNCH-241
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-241
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.7.0
>
>         Attachments: CRUNCH-241.patch
>
>
> Right now, Crunch always writes output files from the "last" stage of whatever kind of job it runs: either the reduce-side of a MapReduce job, or the map-side of a map-only job. This often leads to situations where we have to re-process the same input twice, once for the map-side outputs and again for the reduce-side outputs.
> This change adds the ability for Crunch to write side outputs from the mapper phase of a MapReduce job (i.e., we can write output Targets from both the map side and the reduce side.) This should help lots of pipelines that implement these types of writes execute much faster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira