[jira] [Created] (SPARK-41318) Validate scalability improvements due to JDK-8180450 on Netty 4.1.84.Final

["Francesco Nigro (Jira)" <ji...@apache.org>]

Francesco Nigro created SPARK-41318:
---------------------------------------

             Summary: Validate scalability improvements due to JDK-8180450 on Netty 4.1.84.Final
                 Key: SPARK-41318
                 URL: https://issues.apache.org/jira/browse/SPARK-41318
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.4.0
            Reporter: Francesco Nigro


[https://netty.io/news/2022/10/11/4-1-84-Final.html] has introduced a bug-fix/improvement to prevent a JDK scalability issue due to type check against interfaces (see [https://github.com/netty/netty/pull/12806] for more details - and [https://redhatperf.github.io/post/type-check-scalability-issue/] for a deeper and wider analysis).

it seems that Spark network make uses of Netty channel duplex handlers on [TransportContext|https://github.com/apache/spark/blob/94de3ca2942bb04852510abccf06df1fa8b2dab3/common/network-common/src/main/java/org/apache/spark/network/TransportContext.java#L201] and it's likely to be affected by it (or better, depending which netty version).

Moreover, I suggest to verify if there are other places where the issue can happen and eventually, report them on https://bugs.openjdk.org/browse/JDK-8180450 as well (if they use type checks in an idiomatic form).
The blog post has linked https://github.com/RedHatPerf/type-pollution-agent to detect dangerous candidate code paths.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org