(spark) branch master updated: [SPARK-57430][CORE] Validate length before allocation in `Encoders.ByteArrays.decode`

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 5c29a931066d [SPARK-57430][CORE] Validate length before allocation in `Encoders.ByteArrays.decode`
5c29a931066d is described below

commit 5c29a931066d7bd909f47aea931ce98323e8da27
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Jun 14 08:47:57 2026 -0700

    [SPARK-57430][CORE] Validate length before allocation in `Encoders.ByteArrays.decode`
    
    ### What changes were proposed in this pull request?
    
    `Encoders.ByteArrays.decode()` now validates the length prefix before allocating the array via `Objects.checkFromIndexSize(0, length, buf.readableBytes())`, which requires `0 <= length <= buf.readableBytes()` and throws `IndexOutOfBoundsException` otherwise. After `readInt()`, the reader index is past the prefix, so `readableBytes()` equals the remaining payload bytes.
    
    - Note that we use the Java built-in `Objects.checkFromIndexSize` because we cannot use **Netty's `checkReadableBytes`**. However, both methods throws `IndexOutOfBoundsException` identically for the error cases.
      - https://netty.io/4.2/api/io/netty/buffer/AbstractByteBuf.html#checkReadableBytes(int)
      - https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/Objects.html#checkFromIndexSize(int,int,int)
    
    ### Why are the changes needed?
    
    `decode()` allocated `new byte[length]` from an untrusted length.
    - A negative value throws an opaque `NegativeArraySizeException`.
    - An oversized value can trigger `OutOfMemoryError` on a corrupt or hostile frame.
    
    Validating first fails fast with a clear error.
    
    Note that `OutOfMemoryError` can happen even in a secure environment at the first parse on an unauthenticated channel.
    
    https://github.com/apache/spark/blob/b6450e6765a2fbe307c0b10d744da3ab4583ba44/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthRpcHandler.java#L88-L90
    
    https://github.com/apache/spark/blob/b6450e6765a2fbe307c0b10d744da3ab4583ba44/common/network-common/src/main/java/org/apache/spark/network/crypto/AuthMessage.java#L53-L64
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Pass the CIs with newly added test cases in `EncodersSuite`.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code (Claude Opus 4.8)
    
    Closes #56493 from dongjoon-hyun/SPARK-57430.
    
    Authored-by: Dongjoon Hyun <do...@apache.org>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 .../apache/spark/network/protocol/Encoders.java    |  1 +
 .../spark/network/protocol/EncodersSuite.java      | 22 ++++++++++++++++++++++
 2 files changed, 23 insertions(+)

diff --git a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
index 488e36004f2f..69b7c3159b2c 100644
--- a/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
+++ b/common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java
@@ -107,6 +107,7 @@ public class Encoders {
 
     public static byte[] decode(ByteBuf buf) {
       int length = buf.readInt();
+      Objects.checkFromIndexSize(0, length, buf.readableBytes());
       byte[] bytes = new byte[length];
       buf.readBytes(bytes);
       return bytes;
diff --git a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
index 55b09b66aef6..66299c38fa03 100644
--- a/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
+++ b/common/network-common/src/test/java/org/apache/spark/network/protocol/EncodersSuite.java
@@ -88,4 +88,26 @@ public class EncodersSuite {
     RoaringBitmap[] decodedBitmaps = Encoders.BitmapArrays.decode(buf);
     assertArrayEquals(bitmaps, decodedBitmaps);
   }
+
+  @Test
+  public void testByteArraysEncodeDecode() {
+    byte[] arr = new byte[] { 1, 2, 3, 4, 5 };
+    ByteBuf buf = Unpooled.buffer(Encoders.ByteArrays.encodedLength(arr));
+    Encoders.ByteArrays.encode(buf, arr);
+    assertArrayEquals(arr, Encoders.ByteArrays.decode(buf));
+  }
+
+  @Test
+  public void testByteArraysDecodeShouldFailWhenLengthIsNegative() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(-1);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.ByteArrays.decode(buf));
+  }
+
+  @Test
+  public void testByteArraysDecodeShouldFailWhenLengthExceedsReadableBytes() {
+    ByteBuf buf = Unpooled.buffer();
+    buf.writeInt(Integer.MAX_VALUE);
+    assertThrows(IndexOutOfBoundsException.class, () -> Encoders.ByteArrays.decode(buf));
+  }
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org