You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@crunch.apache.org by "Micah Whitacre (JIRA)" <ji...@apache.org> on 2013/11/20 19:34:36 UTC

[jira] [Commented] (CRUNCH-300) Support reflected Avro record writing from MemPipeline

    [ https://issues.apache.org/jira/browse/CRUNCH-300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827944#comment-13827944 ] 

Micah Whitacre commented on CRUNCH-300:
---------------------------------------

As an FYI, I think there is going to be a conflict between this patch and the proposed changes for CRUNCH-293.

> Support reflected Avro record writing from MemPipeline
> ------------------------------------------------------
>
>                 Key: CRUNCH-300
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-300
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>            Reporter: David Whiting
>            Priority: Minor
>         Attachments: 0001-Allow-MemPipeline-to-write-Avro-files-by-reflection.patch
>
>
> MemPipeline doesn't support writing Avro records via reflection. It seems that this was half implemented but never finished, but I needed it to create some test data to run through a cluster MapReduce test. The current implementation correctly reflects the schema, but then uses a GenericDatumWriter to try and write the record, causing a ClassCastException. The correct way would be to get a ReflectDatumWriter from the ReflectDataFactory.



--
This message was sent by Atlassian JIRA
(v6.1#6144)