[jira] [Assigned] (SPARK-59707) Use ArrayDeque instead of LinkedList in TransportFrameDecoder

["Dongjoon Hyun (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-59707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dongjoon Hyun reassigned SPARK-59707:
-------------------------------------

    Assignee: David Mollitor

> Use ArrayDeque instead of LinkedList in TransportFrameDecoder
> -------------------------------------------------------------
>
>                 Key: SPARK-59707
>                 URL: https://issues.apache.org/jira/browse/SPARK-59707
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 4.1.0
>            Reporter: David Mollitor
>            Assignee: David Mollitor
>            Priority: Minor
>              Labels: pull-request-available
>
> h3. What changes were proposed in this pull request?
> {{TransportFrameDecoder}} sits at the head of the Netty receive pipeline and reassembles length-prefixed frames from incoming socket reads. It keeps the input {{ByteBuf}}s not yet consumed into a frame in a {{buffers}} field, accessed purely as a FIFO queue: appended at the tail on each {{channelRead}}, read/removed from the head as frames are decoded, plus iteration and {{clear}} on cleanup.
> This changes {{buffers}} from {{LinkedList<ByteBuf>}} to {{ArrayDeque<ByteBuf>}}. Every operation used ({{add}}/{{addLast}}, {{getFirst}}, {{removeFirst}}, enhanced-for iteration, {{clear}}) has identical FIFO semantics on {{ArrayDeque}}, so the swap is behavior-preserving. The queue never holds nulls (the only insertion is the non-null {{ByteBuf}} from {{channelRead}}), so {{ArrayDeque}}'s null-rejection is never triggered.
> h3. Why are the changes needed?
> {{LinkedList}} allocates a node object on every {{add}}. {{TransportFrameDecoder.channelRead}} runs on every inbound read of every connection (RPC and block transfer), so the old code produced a steady stream of short-lived list-node allocations on a hot path. {{ArrayDeque}} supports the same add-at-tail / poll-at-head access pattern with a single reused backing array -- the queue is typically just one buffer -- eliminating the per-read node allocation and improving cache locality.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org