You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@crunch.apache.org by "Micah Whitacre (JIRA)" <ji...@apache.org> on 2013/07/23 23:18:48 UTC

[jira] [Commented] (CRUNCH-242) Input/output conversion needs to be controlled by the Source/Target interfaces

    [ https://issues.apache.org/jira/browse/CRUNCH-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13717634#comment-13717634 ] 

Micah Whitacre commented on CRUNCH-242:
---------------------------------------

+1 to the patch.  The changes look fine.  Might make a few people grumpy with the non-passive change of adding new method to interface but nothing that can't be worked around.
                
> Input/output conversion needs to be controlled by the Source/Target interfaces
> ------------------------------------------------------------------------------
>
>                 Key: CRUNCH-242
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-242
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Josh Wills
>         Attachments: CRUNCH-242.patch
>
>
> I was working on adding support for Parquet to Crunch, and ran into the issue that Parquet always assumes that the value it returns is on the "value" side of the key-value pair of an InputFormat/OutputFormat. Crunch, for semi-sensible historical reasons, makes this position dependent on the PTypeFamily (Avro PTypes write to the key, Writable PTypes write to the value). Since the Parquet InputFormat/OutputFormat treat the two types the same way, we need a way for the Source and Target implementations to override the default configuration of the PTypes and choose the right side for the given format.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira