You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2013/12/21 21:18:09 UTC

[jira] [Resolved] (CRUNCH-314) Shuffle and Bundle configuration AvroMode settings clobber one another

     [ https://issues.apache.org/jira/browse/CRUNCH-314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Wills resolved CRUNCH-314.
-------------------------------

       Resolution: Fixed
    Fix Version/s: 0.8.3

Thanks Ryan-- pushed this to master.

> Shuffle and Bundle configuration AvroMode settings clobber one another
> ----------------------------------------------------------------------
>
>                 Key: CRUNCH-314
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-314
>             Project: Crunch
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.9.0
>            Reporter: Ryan Blue
>            Assignee: Josh Wills
>            Priority: Blocker
>             Fix For: 0.10.0, 0.8.3
>
>         Attachments: 0001-CRUNCH-314-Separate-shuffle-and-bundle-AvroMode-conf.patch
>
>
> {{AvroMode}} uses the same configuration property for both shuffle and bundle configuration. If the bundle's AvroMode for a map/reduce round doesn't match the shuffle's AvroMode, then the bundle wins. In some cases, this prevents the shuffle from being able to write data, e.g., because GENERIC is used instead of REFLECT.
> A work-around if to set the strict mode to delegate reader and writer construction to the more permissive one:
> {code:java}
> AvroMode.GENERIC.override(AvroMode.REFLECT);
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)