[GitHub] [trafficserver] keesspoelstra commented on pull request #8534: TS_USE_MMAP: mmap+memcopy instead pread+pwrite

[GitBox <gi...@apache.org>]

keesspoelstra commented on pull request #8534:
URL: https://github.com/apache/trafficserver/pull/8534#issuecomment-1008724337


   @cukiernik , A few percent sounds significant already 👍 this could be very useful. You have run it without RAM cache?
   
   I think that for a write into a memory mapped file it has to page the data into memory first (a read), so that adds some overhead. We may be able to get some more performance here by writing with pwrite instead of memcpy. 
   This also keeps write behaviour predictable, which is better if ATS crashes (which it never does of course). 
   With memory mapped files it is not guaranteed that a valid index will have been written after a crash. A msync could solve this, but the order in which the msync writes the dirty pages is not guaranteed, so a bit more orchestration would be needed for msyncing the index + data. Worst case a lot of data will be written at a very inconvenient moment.
   Linux provides a consistent view between pwrites and memory mapped files, for other systems we'd have to check the behaviour.
   
   You might want to take a look at this:
   https://github.com/apache/trafficserver/blob/3012ca0f94cf7202e75422757d7498bc7e4f9295/iocore/aio/AIO.cc#L598
   I think we're hitting sync I/O here if the pages are not in memory, blocking the thread instead of using the original async nature of the function. I don't know if passing the read to a number of specialized threads would yield better performance, I suspect it will all depend on the workload. 
   Mincore and msync can check whether pages are in memory, so that could be used to set up a fast (serve from memory) path, but the drawback would be the overhead of these functions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org