You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@avro.apache.org by "Howard Tong (JIRA)" <ji...@apache.org> on 2018/07/06 05:31:00 UTC

[jira] [Created] (AVRO-2197) add a new constructor in DataFileReader & DataFileWriter with passing in Schema object

Howard Tong created AVRO-2197:
---------------------------------

             Summary: add a new constructor in DataFileReader & DataFileWriter with passing in Schema object
                 Key: AVRO-2197
                 URL: https://issues.apache.org/jira/browse/AVRO-2197
             Project: Avro
          Issue Type: Improvement
          Components: java
    Affects Versions: 1.8.2
            Reporter: Howard Tong


When processing big number of small files (with the same schema header), reading/writing each file need to read/write the head and parsing the json script, which is very slow. 

If adding a new constructor in DataFileReader and DataFileWriter that allows pass in already parsed Schema object/script,  then it will greatly improve the reading/writing performance for such cases. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)