You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/08/26 13:40:26 UTC

[GitHub] [hive] gabrywu opened a new pull request #1434: [HIVE-16352] add ability to AvroGenericRecordReader to skip invalid sync blocks

gabrywu opened a new pull request #1434:
URL: https://github.com/apache/hive/pull/1434


   ### What changes were proposed in this pull request?
   1. add AvroGenericRecordReader.nextRecord
   2. optimize AvroGenericRecordReader.next adding ability to skip invalid sync blocks
   3. add enum value AVRO_SERDE_ERROR_SKIP to AvroSerdeUtils.AvroTableProperties
   
   ### Why are the changes needed?
   
   when reading the Avro file which has a bad file format in Hive, we want to skip the invalid sync errors simply
   https://issues.apache.org/jira/browse/HIVE-16352
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   NO. The default value of AVRO_SERDE_ERROR_SKIP is false keeping the original logic
   
   ### How was this patch tested?
   
   add unit test cases in TestAvroGenericRecordReader.class
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org

[GitHub] [hive] gabrywu closed pull request #1434: [HIVE-16352] add ability to AvroGenericRecordReader to skip invalid sync blocks

Posted by GitBox <gi...@apache.org>.
gabrywu closed pull request #1434:
URL: https://github.com/apache/hive/pull/1434


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org