You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by do...@apache.org on 2026/06/14 19:11:50 UTC

(spark) branch branch-4.x updated: [SPARK-57439][CORE] Validate length before allocation in `Encoders.(Int|Long)Arrays` decode

This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 5087cbefbb32 [SPARK-57439][CORE] Validate length before allocation in `Encoders.(Int|Long)Arrays` decode
5087cbefbb32 is described below

commit 5087cbefbb32358a403695b4811ae9216c216809
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Jun 14 12:11:32 2026 -0700

    [SPARK-57439][CORE] Validate length before allocation in `Encoders.(Int|Long)Arrays` decode
    
    ### What changes were proposed in this pull request?
    
    `Encoders.IntArrays.decode()` and `Encoders.LongArrays.decode()` now validate the
    element-count prefix before allocating the array.
    
    Unlike the 1-byte-element `Encoders.Strings` / `Encoders.ByteArrays` cases
    (SPARK-57426, SPARK-57430), where the byte length maps 1:1 to readable bytes, each
    element here occupies a fixed multiple of bytes, so the upper bound is divided by the
    element size:
    
    - `IntArrays`: `Objects.checkFromIndexSize(0, numInts, buf.readableBytes() / 4)` (4 bytes per `int`)
    - `LongArrays`: `Objects.checkFromIndexSize(0, numLongs, buf.readableBytes() / 8)` (8 bytes per `long`)
    
    This requires `0 <= count <= (remaining bytes / element size)` and throws
    `IndexOutOfBoundsException` otherwise. After `readInt()`, the reader index is past the
    count prefix, so `buf.readableBytes() / elementSize` is the maximum number of elements
    actually decodable from the remaining payload.
    
    - Note that we use the Java built-in `Objects.checkFromIndexSize` because we cannot use
      **Netty's `checkReadableBytes`**. However, both methods throw `IndexOutOfBoundsException`
      identically for the error cases.
      - https://netty.io/4.2/api/io/netty/buffer/AbstractByteBuf.html#checkReadableBytes(int)
      - https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/Objects.html#checkFromIndexSize(int,int,int)
    
    ### Why are the changes needed?
    
    `decode()` allocated `new int[numInts]` / `new long[numLongs]` from an untrusted count prefix.
    - A negative value throws an opaque `NegativeArraySizeException`.
    - An oversized value can trigger `OutOfMemoryError` on a corrupt or hostile frame.
    
    Validating first fails fast with a clear error, consistent with SPARK-57426 (`Strings`)
    and SPARK-57430 (`ByteArrays`).
    
    These two decoders are used by the shuffle wire-protocol messages, e.g.
    `FetchShuffleBlocks`, `FetchShuffleBlockChunks`, `MergeStatuses`, and `LocalDirsForExecutors`.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Pass the CIs.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Claude Opus 4.8)
    
    Closes #56500 from dongjoon-hyun/SPARK-57439.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
    (cherry picked from commit 6330518717251fa34cc6e93abf254b3f2c368f7b)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../apache/spark/network/protocol/Encoders.java    |  2 +
 .../spark/network/protocol/EncodersSuite.java      | 44 ++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
index 69b7c3159b2c..d69ad7c1f9ce 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
@@ -156,6 +156,7 @@ public class Encoders {
 
     public static int[] decode(ByteBuf buf) {
       int numInts = buf.readInt();
+      Objects.checkFromIndexSize(0, numInts, buf.readableBytes() / 4);
       int[] ints = new int[numInts];
       for (int i = 0; i < ints.length; i ++) {
         ints[i] = buf.readInt();
@@ -179,6 +180,7 @@ public class Encoders {
 
     public static long[] decode(ByteBuf buf) {
       int numLongs = buf.readInt();
+      Objects.checkFromIndexSize(0, numLongs, buf.readableBytes() / 8);
       long[] longs = new long[numLongs];
       for (int i = 0; i < longs.length; i ++) {
         longs[i] = buf.readLong();
diff --git a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
index 66299c38fa03..0a07ebf17ded 100644
--- a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
+++ b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
@@ -110,4 +110,48 @@ public class EncodersSuite {
     buf.writeInt(Integer.MAX_VALUE);
     assertThrows(IndexOutOfBoundsException.class, () -> Encoders.ByteArrays.decode(buf));
   }
+
+  @Test
+  public void testIntArraysEncodeDecode() {
+    int[] arr = new int[] { 1, 2, 3, 4, 5 };
+    ByteBuf buf = Unpooled.buffer(Encoders.IntArrays.encodedLength(arr));
+    Encoders.IntArrays.encode(buf, arr);
+    assertArrayEquals(arr, Encoders.IntArrays.decode(buf));
+  }
+
+  @Test
+  public void testIntArraysDecodeShouldFailWhenLengthIsNegative() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(-1);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.IntArrays.decode(buf));
+  }
+
+  @Test
+  public void testIntArraysDecodeShouldFailWhenLengthExceedsReadableBytes() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(Integer.MAX_VALUE);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.IntArrays.decode(buf));
+  }
+
+  @Test
+  public void testLongArraysEncodeDecode() {
+    long[] arr = new long[] { 1L, 2L, 3L, 4L, 5L };
+    ByteBuf buf = Unpooled.buffer(Encoders.LongArrays.encodedLength(arr));
+    Encoders.LongArrays.encode(buf, arr);
+    assertArrayEquals(arr, Encoders.LongArrays.decode(buf));
+  }
+
+  @Test
+  public void testLongArraysDecodeShouldFailWhenLengthIsNegative() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(-1);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.LongArrays.decode(buf));
+  }
+
+  @Test
+  public void testLongArraysDecodeShouldFailWhenLengthExceedsReadableBytes() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(Integer.MAX_VALUE);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.LongArrays.decode(buf));
+  }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org