You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "dongjoon-hyun (via GitHub)" <gi...@apache.org> on 2026/06/13 21:11:29 UTC
[PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
dongjoon-hyun opened a new pull request, #56488:
URL: https://github.com/apache/spark/pull/56488
### What changes were proposed in this pull request?
`Encoders.Strings.decode()` now validates the length prefix before allocating the array, requiring `0 <= length <= buf.readableBytes()` and throwing `IllegalArgumentException` otherwise. After `readInt()`, the reader index is past the prefix, so `readableBytes()` equals the remaining payload bytes.
### Why are the changes needed?
`decode()` allocated `new byte[length]` from an untrusted length. A negative value throws an opaque `NegativeArraySizeException`, and an oversized value can trigger `OutOfMemoryError` on a corrupt or hostile frame. Validating first fails fast with a clear error.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added round-trip, negative-length, and length-exceeds-readable-bytes tests in `EncodersSuite`, verified with `build/sbt 'network-common/testOnly *EncodersSuite'`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56488:
URL: https://github.com/apache/spark/pull/56488#issuecomment-4700157442
Thank you, @huaxingao .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #56488:
URL: https://github.com/apache/spark/pull/56488#discussion_r3408661557
##########
common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java:
##########
@@ -41,6 +42,7 @@ public static void encode(ByteBuf buf, String s) {
public static String decode(ByteBuf buf) {
int length = buf.readInt();
+ Objects.checkIndex(length, buf.readableBytes() + 1);
Review Comment:
FYI, `buf.readableBytes() + 1` is safe from integer overflow because `buf.readInt()` succeeded already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on code in PR #56488:
URL: https://github.com/apache/spark/pull/56488#discussion_r3408661557
##########
common/network-common/src/main/java/org/apache/spark/network/protocol/Encoders.java:
##########
@@ -41,6 +42,7 @@ public static void encode(ByteBuf buf, String s) {
public static String decode(ByteBuf buf) {
int length = buf.readInt();
+ Objects.checkIndex(length, buf.readableBytes() + 1);
Review Comment:
FYI, `buf.readableBytes() + 1` is safe from integer overflow because `buf.readInt()` succeeded already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56488:
URL: https://github.com/apache/spark/pull/56488#issuecomment-4700161259
Merged to master/4.x for now because we have Apache Spark 4.2.0 RC3 vote currently.
If it fails, I will consider to cherry-pick this to `branch-4.2` and older.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56488:
URL: https://github.com/apache/spark/pull/56488#issuecomment-4700086772
cc @huaxingao, FYI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun closed pull request #56488: [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode`
URL: https://github.com/apache/spark/pull/56488
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57426][CORE] Validate length before allocation in `Encoders.Strings.decode` [spark]
Posted by "dongjoon-hyun (via GitHub)" <gi...@apache.org>.
dongjoon-hyun commented on PR #56488:
URL: https://github.com/apache/spark/pull/56488#issuecomment-4700091264
`core` module passed already.
<img width="613" height="77" alt="Screenshot 2026-06-13 at 16 22 15" src="https://github.com/user-attachments/assets/331189c1-fe3b-4589-aeff-5b4e2c281a31" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org