You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "fallintoplace (via GitHub)" <gi...@apache.org> on 2026/07/25 16:05:57 UTC
[PR] fix(parquet/encoding): reject invalid delta length payloads [arrow-go]
fallintoplace opened a new pull request, #1013:
URL: https://github.com/apache/arrow-go/pull/1013
### Rationale for this change
The DELTA_LENGTH_BYTE_ARRAY decoder currently ignores errors while decoding its length stream and slices the payload with those unchecked lengths. Negative lengths or lengths beyond the available payload therefore panic instead of returning a decoding error.
### What changes are included in this PR?
- Propagate errors and short reads from the delta length stream.
- Check that the encoded length count fits the page value count.
- Reject negative lengths and cumulative lengths beyond the suffix payload.
### Are these changes tested?
Yes. Focused tests cover negative and oversized lengths, and the existing round-trip tests continue to pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(parquet/encoding): reject invalid delta length payloads [arrow-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #1013:
URL: https://github.com/apache/arrow-go/pull/1013#issuecomment-5094392637
Reviewed and the analysis holds up: `DeltaLengthByteArrayDecoder.Decode` indexes `d.lengths[i]` with `nvals` derived from the *logical* count, so a nullable page with two logical positions and one physical value reads past the end of the length table. Switching to the encoded physical count plus the negative/oversized length validation is the right fix.
I just merged #1014, which rewrites the same `DeltaByteArrayDecoder.SetData` this PR touches, so this now needs a rebase onto `main` — the hunks overlap directly. Once you rebase and CI is green I will merge this. Note that #1014 already added the `lastVal` reset and the prefix-offset bounds check, so some of the overlap should collapse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(parquet/encoding): reject invalid delta length payloads [arrow-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #1013:
URL: https://github.com/apache/arrow-go/pull/1013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org