[jira] Commented: (PIG-1745) Disable converting bytes loading from BinStorage

["Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966289#action_12966289 ] 

Dmitriy V. Ryaboy commented on PIG-1745:
----------------------------------------

Daniel,
check out how I addressed a very similar problem in the HBase loader -- I have a default caster, and allow a user to specify one using a constructor if necessary. I think that's cleaner than adding and extra storage class.

https://github.com/apache/pig/blob/trunk/src/org/apache/pig/backend/hadoop/hbase/HBaseStorage.java

> Disable converting bytes loading from BinStorage
> ------------------------------------------------
>
>                 Key: PIG-1745
>                 URL: https://issues.apache.org/jira/browse/PIG-1745
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.9.0
>
>         Attachments: PIG-1745-1.patch, PIG-1745-2.patch
>
>
> If we load bytes from BinStorage, we don't actually know how we get these bytes originally, and we will not have a way to cast those bytes. Ideally we shall encode caster into BinStorage data file, but we are not there yet. Currrently bytesToXXX methods for BinStorage is wrong and it results unexpected errors. Eg.
> {code}
> a = load '1.txt' as (a0, a1, a2);
> store a into '1.bin' as BinStorage();
> a = load '1.bin' using BinStorage as (a0, a1, a2);
> b = foreach a generate (long)a0;
> dump b;
> {code}
> The code will run but produce wrong data. It's less confusing if we throw an exception in this case.
> Release Notes:
> Pig will throw exception in the case we want to convert bytes loading from BinStorage

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.