[GitHub] [kafka] emissionnebula commented on pull request #13437: KAFKA-14828: Remove R/W locks using persistent data structures

["emissionnebula (via GitHub)" <gi...@apache.org>]

emissionnebula commented on PR #13437:
URL: https://github.com/apache/kafka/pull/13437#issuecomment-1493821132

   > Since you mentioned, "authorizer is quite read-heavy with very infrequent writes", and given the above risk highlighted above, would having slow writes with CopyOnWrite be an acceptable trade-off?
   
   There is another [PR](https://github.com/apache/kafka/pull/13280) by @rondagostino that evaluates these libraries for the Kraft metadata image change performance. That code currently uses the CopyOnWrite approach. The performance of these persistent libraries over existing code is multi-magnitude better - CopyOnWrite = ~31sec/op and PCollection = 509 ns/op. 
   
   > let's say we end up using a dependency that is maintained by a single person and they decide to stop maintaining it.
   
   I totally agree with this risk but the gain with these libraries is quite significant so maybe we can consider this also as a trade-off.
   
   > That leaves us with a choice to either overhaul our code base and remove all usages of that library (which I expect to be very intrusive for locking data structures) or take over ownership of something that we as a community are not experts in.
   
   We can probably minimize the risk of changing the code at all usage points by defining interfaces and wrapper classes in our code to standardize the usage of persistent collections. In the future, if we decide to move away from one library to another then it wouldn't require changing the code everywhere.
   
   cc - @ijuma @mimaison @cmccabe @omkreddy @dajac, what are your views on this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org