[GitHub] [incubator-daffodil] olabusayoT edited a comment on pull request #453: Add SAX Unparse Event Batching

[GitBox <gi...@apache.org>]

olabusayoT edited a comment on pull request #453:
URL: https://github.com/apache/incubator-daffodil/pull/453#issuecomment-724944482


   @stevedlawrence As far as performance, see below. I tested it with a baseline of ebd1af445b9c9892481d50bdd2f094eec4f503db with 3 runs for each setting.
   
   It was run against a CSV file with 1000 entires and the CSV/src/main/resources/com/tresys/csv/xsd/csvHeaderEnforced.dfdl.xsd  schemas. N was set to 1000 for each run.
   
   No Batching (SAX)
   
   total unparse time (sec): 119.045022
   min rate (files/sec): 0.887929
   max rate (files/sec): 10.849576
   avg rate (files/sec): 8.400183
   
   total unparse time (sec): 122.495742
   min rate (files/sec): 0.933206
   max rate (files/sec): 10.572712
   avg rate (files/sec): 8.163549
   
   total unparse time (sec): 115.577783
   min rate (files/sec): 0.887943
   max rate (files/sec): 10.906708
   avg rate (files/sec): 8.652182
   
   BatchSize = 1
   
   total unparse time (sec): 123.355449
   min rate (files/sec): 0.946018
   max rate (files/sec): 11.109970
   avg rate (files/sec): 8.106654
   
   total unparse time (sec): 113.578335
   min rate (files/sec): 0.942423
   max rate (files/sec): 11.725798
   avg rate (files/sec): 8.804496
   
   total unparse time (sec): 118.298859
   min rate (files/sec): 1.013238
   max rate (files/sec): 10.598986
   avg rate (files/sec): 8.453167
   
   BatchSize = 10
   
   total unparse time (sec): 37.720906
   min rate (files/sec): 1.087529
   max rate (files/sec): 41.515119
   avg rate (files/sec): 26.510498
   
   total unparse time (sec): 37.424359
   min rate (files/sec): 1.296234
   max rate (files/sec): 40.985226
   avg rate (files/sec): 26.720564
   
   total unparse time (sec): 36.939181
   min rate (files/sec): 1.272301
   max rate (files/sec): 42.095508
   avg rate (files/sec): 27.071526
   
   Batch Size = 100
   
   total unparse time (sec): 27.516491
   min rate (files/sec): 1.102611
   max rate (files/sec): 55.597572
   avg rate (files/sec): 36.341843
   
   total unparse time (sec): 29.054037
   min rate (files/sec): 1.181798
   max rate (files/sec): 53.005364
   avg rate (files/sec): 34.418625
   
   total unparse time (sec): 27.553790
   min rate (files/sec): 1.364561
   max rate (files/sec): 55.164476
   avg rate (files/sec): 36.292648
   
   Batch Size = 1000
   
   total unparse time (sec): 24.731927
   min rate (files/sec): 1.370080
   max rate (files/sec): 59.091401
   avg rate (files/sec): 40.433565
   
   total unparse time (sec): 25.288054
   min rate (files/sec): 1.389677
   max rate (files/sec): 62.957730
   avg rate (files/sec): 39.544364
   
   total unparse time (sec): 25.620273
   min rate (files/sec): 1.297336
   max rate (files/sec): 59.002985
   avg rate (files/sec): 39.031590
   
   
   
   
    


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org