You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@avro.apache.org by Johannes Schulte <jo...@gmail.com> on 2018/08/07 09:30:55 UTC

Mapreduce old vs new API: Ignoring non-avro files

Hi,

when using a directory with avro files with an Avro Hadoop InputFormat, do
I see it correctly that the old API implementation (in the mapred-package)
respects the property

avro.mapred.ignore.inputs.without.extension

(which defaults to true) using this code path
https://github.com/apache/avro/blob/cf2f30336efe0ecc3debc7bede86fde6d23f7c79/lang/java/mapred/src/main/java/org/apache/avro/mapred/AvroInputFormat.java#L54

while the new API (under mapreduce) that uses the term AvroKeyInputFormat
for the same use case ignores this property as it does not override the
default implementation of FileInputFormat?

https://github.com/apache/avro/blob/cf2f30336efe0ecc3debc7bede86fde6d23f7c79/lang/java/mapred/src/main/java/org/apache/avro/mapreduce/AvroKeyInputFormat.java#L40








Is this worth filing a pull request?

Cheers,

Johannes