You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Ashan Taha <at...@currenex.com> on 2016/10/20 12:00:14 UTC

Ensuring an Avro File is NOT Splitable

Hi

What's the best way to make sure an Avro file is NOT Splitable when read in Spark?
Would you override the AvroKeyInputFormat.issplitable (to return false) and then call this using newAPIHadoopRDD? Or is there a better way using the sqlContext.read?

Thanks in advance