[jira] [Commented] (CRUNCH-199) Add Source and Target Support for AvroTrevniKey files

["Josh Wills (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CRUNCH-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644246#comment-13644246 ] 

Josh Wills commented on CRUNCH-199:
-----------------------------------

Hey Micah-- the thing that moves the data from the temporary location that is created during the crunch job run to the final location is the MSCROutputHandler in o.a.c.impl.mr.plan. I think the source of the issue is at the end of the configureMapReduce function in the TrevniKeyTarget, where you reset the outputPath. I think that this line should go.

Also, I agree with your comment: the TrevniKeyOutputFormat isn't aware of multiple schema outputs within the context of a single job, so you'll need to use a FormatBundle directly so that different config schemas don't conflict with each other. I'll post a patch to demonstrate.


                
> Add Source and Target Support for AvroTrevniKey files
> -----------------------------------------------------
>
>                 Key: CRUNCH-199
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-199
>             Project: Crunch
>          Issue Type: Wish
>          Components: Core, IO
>            Reporter: Micah Whitacre
>            Assignee: Josh Wills
>            Priority: Minor
>         Attachments: CRUNCH-199.patch
>
>
> Add Source, Target, and SourceTarget, support for AvroTrevniKeyInputFormat[1] and AvroTrevniKeyOutputFormat[2].
> [1] - http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyInputFormat.html
> [2] - http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyOutputFormat.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira