You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Mridul Muralidharan (Jira)" <ji...@apache.org> on 2022/11/04 19:37:00 UTC

[jira] [Resolved] (SPARK-40950) isRemoteAddressMaxedOut performance overhead on scala 2.13

     [ https://issues.apache.org/jira/browse/SPARK-40950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mridul Muralidharan resolved SPARK-40950.
-----------------------------------------
    Fix Version/s: 3.4.0
       Resolution: Fixed

Issue resolved by pull request 38427
[https://github.com/apache/spark/pull/38427]

> isRemoteAddressMaxedOut performance overhead on scala 2.13
> ----------------------------------------------------------
>
>                 Key: SPARK-40950
>                 URL: https://issues.apache.org/jira/browse/SPARK-40950
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 3.2.2, 3.3.1
>            Reporter: Emil Ejbyfeldt
>            Assignee: Emil Ejbyfeldt
>            Priority: Major
>             Fix For: 3.4.0
>
>
> On scala 2.13 the blocks in FetchRequest is sometimes backed by a `List` while in 2.12 it would be ArrayBuffer. This means that calculating the length of the blocks which is done in isRemoteAddressMaxedOut and other places now much more expensive.  This is because in 2.13 `Seq` is can no longer be backed by a mutable collection.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org