You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Russell Jurney <ru...@gmail.com> on 2013/01/07 04:38:35 UTC
Sync() between records? How do we recover from a bad record, using DataFileReader?
We are trying to recover, report bad record, and move to the next record of
an Avro file in PIG-3015 and PIG-3059. It seems that sync blocks don't
exist between files, however.
How should we recover from a bad record using Avro's DataFileReader?
https://issues.apache.org/jira/browse/PIG-3015
https://issues.apache.org/jira/browse/PIG-3059
Russell Jurney http://datasyndrome.com
Re: Sync() between records? How do we recover from a bad record,
using DataFileReader?
Posted by Scott Carey <sc...@apache.org>.
For the corruption test, try corrupting the records, not the sync marker.
The features added to DataFileReader for corruption recovery were for the
case when decoding a record fails (corrupted record), not for when a sync
marker is corrupted. Perhaps we should add that too, but it does not
surprise me that that case has a bug.
On 1/6/13 7:38 PM, "Russell Jurney" <ru...@gmail.com> wrote:
>We are trying to recover, report bad record, and move to the next record
>of an Avro file in PIG-3015 and PIG-3059. It seems that sync blocks don't
>exist between files, however.
>
>How should we recover from a bad record using Avro's DataFileReader?
>
>https://issues.apache.org/jira/browse/PIG-3015
>https://issues.apache.org/jira/browse/PIG-3059
>
>Russell Jurney http://datasyndrome.com