You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/11/04 14:37:51 UTC

[GitHub] [flink] JingGe commented on pull request #17520: [FLINK-24565][avro] Port avro file format factory to BulkReaderFormatFactory

JingGe commented on pull request #17520:
URL: https://github.com/apache/flink/pull/17520#issuecomment-961072361


   @tsreaper many thanks for your effort and for sharing the benchmark data. 
   
   The option of using BulkFormat + ArrayList is almost the same as using StreamFormat+StreamFormatAdapter, except the memory size control. Have you try to control the number of records each batchRead() will fetch instead of fetch all records (code reference see [here](https://github.com/tsreaper/flink/commit/3b86337cea499cd4245a34550a6b597239be3066#diff-07c21eca7aca500ba4675ecd3ace539cf31d69797fd254b7b914d22198a789baR145-R148))of the current block in one shot?
   
   For the option of Stream Format based on Stephan's draft, may I know how you controlled the `StreamFormat.FETCH_IO_SIZE`? Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org