[GitHub] davisp opened a new pull request #806: FEEDBACK ONLY: Compactor optimize emsort

[gi...@git.apache.org]

davisp opened a new pull request #806: FEEDBACK ONLY: Compactor optimize emsort
URL: https://github.com/apache/couchdb/pull/806
 
 
   This PR is for feedback only. We've got PSE almost ready to merge so this will need to be slightly re-worked for that case which should theoretically cut down on the size of the change since the couch_bt_engine already contains most of the big diffs trying to clean up the compactor code so it can be worked on.
   
   The crux of this optimization is to add the new append_terms and pread_terms APIs to couch_file which allows us to read and write multiple iolists to a couch_file simultaneously. This is quite helpful for things like emsort that used to write large terms to disk and then have to copy them multiple times during its decimation phase. With the multi-IO APIs we can write them all at once and then just use the file offsets during the sort phases. Attempting to do the same approach using append_term/pread_term introduces significant amounts of time to compaction due to the increased number of calls through couch_file.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services