[jira] Resolved: (PIG-1745) Disable converting bytes loading from BinStorage

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-1745.
-----------------------------

      Resolution: Fixed
    Release Note: 
1. Converting bytes loading from BinStorage() will now result an error.
2. If user clearly knows the caster for the BinStorage, he/she can pass the caster class name to the construct of BinStorage. For example, if data is load from PigStorage (or other LoadFunc using Utf8StorageConverter) and store using BinStorage, user can pass "org.apache.pig.builtin.Utf8StorageConverter" to  the construct of BinStorage (or pass "Utf8StorageConverter", since we will search for "org.apache.pig.builtin" package by default). By doing this, converting bytes to other type will still work.

  was:
1. Converting bytes loading from BinStorage() will now result an error.
2. If user clearly understand that the data is load from PigStorage (or other LoadFunc using Utf8StorageConverter), he/she should use BinStorageWithCaster. By doing this, converting bytes to other type will still work.

    Hadoop Flags: [Reviewed]

test-patch:
[exec] +1 overall. 
[exec] 
[exec] +1 @author. The patch does not contain any @author tags.
[exec] 
[exec] +1 tests included. The patch appears to include 2 new or modified tests.
[exec] 
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] 
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] 
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] 
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

Unit test:
pass

end-to-end test:
pass

> Disable converting bytes loading from BinStorage
> ------------------------------------------------
>
>                 Key: PIG-1745
>                 URL: https://issues.apache.org/jira/browse/PIG-1745
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.9.0
>
>         Attachments: PIG-1745-1.patch, PIG-1745-2.patch, PIG-1745-3.patch
>
>
> If we load bytes from BinStorage, we don't actually know how we get these bytes originally, and we will not have a way to cast those bytes. Ideally we shall encode caster into BinStorage data file, but we are not there yet. Currrently bytesToXXX methods for BinStorage is wrong and it results unexpected errors. Eg.
> {code}
> a = load '1.txt' as (a0, a1, a2);
> store a into '1.bin' as BinStorage();
> a = load '1.bin' using BinStorage as (a0, a1, a2);
> b = foreach a generate (long)a0;
> dump b;
> {code}
> The code will run but produce wrong data. It's less confusing if we throw an exception in this case.
> Release Notes:
> Pig will throw exception in the case we want to convert bytes loading from BinStorage

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.