Re: [PR] Workarount to continue with log4j-2.17.0 by using slf4j18 [pinot]

["gortiz (via GitHub)" <gi...@apache.org>]

gortiz commented on PR #11909:
URL: https://github.com/apache/pinot/pull/11909#issuecomment-1792184336

   Let me summarize the current state:
   - We wanted to be able to execute Pinot in Java 21.
   - Spark 3.2 does not support Java 21, so I updated it to Spark 3.5
   - Spark 3.5 does uses slf4j v2.0.
   - Log4j2 v2.17.1 does not support slf4j v2.0.
   - I updated log4j2 to v2.20.0, which supports slf4j v2.0
   - LinkedIn reports they have seen that newer versions of log4j2 have some thread contention
   - @mcvsubbu tried to revert log4j to 2.17.x family without success due to runtime errors in Spark 3.x Pinot plugin.
   
   I don't think there is a way to keep Spark 3.5 (and therefore Java 21 support) without upgrading log4j2. What I've asked in [another PR](https://github.com/apache/pinot/pull/11702#issuecomment-1784675061) is whether this thread contention problem has been reported to log4j2 maintainers. I've made a fast search and didn't find issues related to that in their repository.
   
   I think the best solution would be to solve the thread contention issue in log4j2. We don't need to actually fix it our self, but it would be great if you could report the issue in their repo so they know it and therefore may fix it. As said in the other thread, we don't want to keep using older versions of compatible libraries due to perf regressions. That is a dangerous path that can end up having a vulnerable dependency we cannot upgrade.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org