[jira] [Updated] (CRUNCH-71) PType mapping functions are not initialized before being used for deep copying

["Gabriel Reid (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CRUNCH-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gabriel Reid updated CRUNCH-71:
-------------------------------

    Attachment: CRUNCH-71.patch

Patch to resolve the issue attached.

I'm not 100% happy with this solution, as I would prefer that the PType would be supplied to the DoFn at runtime instead of the DoFn being responsible for calling PType#initialize.

However, that approach could bring a lot of extra work along with it in job setup as there is not a 1-to-1 relationship between DoFns and PTypes. 

As object reuse issues are pretty isolated in MR contexts (joins are the main place where I see them occurring) then this fix feels ok to me for now. Any objections to this patch?
                
> PType mapping functions are not initialized before being used for deep copying
> ------------------------------------------------------------------------------
>
>                 Key: CRUNCH-71
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-71
>             Project: Crunch
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Gabriel Reid
>            Assignee: Gabriel Reid
>         Attachments: CRUNCH-71.patch
>
>
> The PType#getDetachedValue method performs a deep copy (if needed) in order to allow DoFns to hold on to values that have been passed through them (for example, in join functions).
> The WritablePType class uses the built-in input and output MapFns in the PType to handle this deep copying, but the input and output MapFns don't get initialized (i.e. initialize isn't called on them) after they are deserialized along with the DoFn that is using them. In some rare cases (at least for tuples), this can result in NullPointerExceptions or other nastiness.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira