[jira] [Assigned] (PIG-2614) AvroStorage crashes on LOADING a single bad error

["Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-2614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy V. Ryaboy reassigned PIG-2614:
--------------------------------------

    Assignee: Jonathan Coveney

Looks like Jon's all over this.. assigning to him.

Would be nice to make this a global setting (and punt per-loader settings to ONERROR implementation)

But that's outside the scope of this ticket, let's at least confirm this works.

Russell, if you can post input that knocks you to the wrong offset, that could help with reproducing the error you are getting with the current patch.
                
> AvroStorage crashes on LOADING a single bad error
> -------------------------------------------------
>
>                 Key: PIG-2614
>                 URL: https://issues.apache.org/jira/browse/PIG-2614
>             Project: Pig
>          Issue Type: Bug
>          Components: piggybank
>    Affects Versions: 0.10.0, 0.11
>            Reporter: Russell Jurney
>            Assignee: Jonathan Coveney
>              Labels: avro, avrostorage, bad, book, cutting, doug, for, my, pig, sadism
>             Fix For: 0.11, 0.10.1
>
>         Attachments: PIG-2614_0.patch, PIG-2614_1.patch
>
>
> AvroStorage dies when a single bad record exists, such as one with missing fields.  This is very bad on 'big data,' where bad records are inevitable.  See discussion at http://www.quora.com/Big-Data/In-Big-Data-ETL-how-many-records-are-an-acceptable-loss for more theory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira