[GitHub] [accumulo] ddanielr commented on issue #3198: FileManager.filePermits semaphore is unfair, blocking scans under high load

["ddanielr (via GitHub)" <gi...@apache.org>]

ddanielr commented on issue #3198:
URL: https://github.com/apache/accumulo/issues/3198#issuecomment-1452508034

   From the [java docs](https://devdocs.io/openjdk~11/java.base/java/util/concurrent/semaphore): 
   ```
   Generally, semaphores used to control resource access should be initialized as fair, to ensure that no thread is starved out from accessing a resource. When using semaphores for other kinds of synchronization control, the throughput advantages of non-fair ordering often outweigh fairness considerations.
   ```
   Thread starvation for scans seem extremely problematic in a high load environment. Especially so once you add in expected behavior of the Scan Executors thread pools.
   
   A fair semaphore ensures this can't happen by enforcing a FIFO queue for resource requests once all of the available file permits are reserved. 
   
   This code is pretty stable between 1.10 and 2.1.x so given the impact to scans, this bug should be fixed in the 1.10.x line. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org