You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by "chaokunyang (via GitHub)" <gi...@apache.org> on 2026/08/02 23:44:48 UTC
[PR] feat: bound unbacked container deserialization work [fory]
chaokunyang opened a new pull request, #3910:
URL: https://github.com/apache/fory/pull/3910
## Why?
Count-driven collection, map, and compatible-skip reads can legally consume fewer bytes than items when element codecs have empty bodies. Without independent work accounting, a small input can force disproportionate CPU work or repeated skip iterations, and count-derived capacity can be allocated without proportional backing bytes.
## What does this PR do?
- Adds a root-scoped `maxUnbackedContainerItems` allowance, defaulting to 8192, across Java, C++, Rust, Swift, Python/Cython, Go, JavaScript, C#, and Dart.
- Settles collection shortfall every 1024 items and map shortfall at existing chunk boundaries, with final-tail settlement and root-only failure cleanup.
- Gates count-derived allocation using readable bytes plus the remaining allowance, while preserving existing graph-memory accounting.
- Uses compile-time, generated, or resolved serializer progress facts so proven-positive collection/map paths keep the prior loop shape and do not access the new budget or cursor state.
- Keeps custom, dynamic, unknown, and compatible-skip bodies conservative; rejects zero map chunks only where the owning loop would otherwise make no progress.
- Adds focused root-reset, allocation, stream-compaction, generated-code, static-fast-path, map-chunk, and compatible-skip tests plus public/security documentation.
- Does not change the wire encoding or RefReader/RefWriter architecture.
## Related issues
None.
## AI Contribution Checklist
- [x] Substantial AI assistance was used in this PR: `yes`
- [x] I included the completed checklist and AI Usage Disclosure below.
- [x] I can explain and defend all important changes without AI help.
- [x] I reviewed the AI-assisted code changes line by line before submission.
- [x] I completed self-review and fixed issues before requesting AI review.
- [x] Two fresh AI reviewers reviewed final HEAD `e42ab7a3624e8e1bf16c9c6ce379713fe61e1465`: one Fory-guided full-diff reviewer and one independent general reviewer.
- [x] All actionable comments were addressed and both final reviewers reported no actionable findings.
- [x] Final persisted review-artifact links will be added immediately after PR creation.
- [x] Relevant local verification was run and reviewed.
- [x] Tests and documentation were updated.
- [x] Apache-2.0-compatible provenance was confirmed; no incompatible third-party code was introduced.
### AI Usage Disclosure
- substantial_ai_assistance: yes
- scope: design refinement, cross-runtime implementation, tests, documentation, self-review, and fix iteration
- affected_files_or_subsystems: collection/map/compatible-skip readers, read contexts, generated/static serializer metadata, configuration, tests, and deserialization documentation across all supported runtimes
- ai_review: line-by-line self-review completed; the Fory-guided review found two hot-path issues, both accepted portions were fixed and inner-reviewed; a fresh full-diff Fory reviewer and a separate independent general reviewer then both returned `No actionable findings.`
- ai_review_artifacts: persisted PR-comment links will be inserted immediately after PR creation
- human_verification: focused and full runtime suites were reviewed, including Java (287 focused), Kotlin (88 runtime + 24 KSP), Scala (108 with 2 ignored), C++ (five focused targets, 123 tests), Rust workspace checks/tests, Swift full package (267), Python/Cython, Go, JavaScript, C#, Dart, formatting/static checks, and relevant xlang suites
- performance_verification: no benchmark was run per explicit maintainer instruction; positive generated/static paths are structurally free of budget/cursor accounting, with generated-code and focused fast-path assertions
- provenance_license_confirmation: Apache-2.0-compatible provenance confirmed; no incompatible third-party code introduced
## Does this PR introduce any user-facing change?
- [x] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
The public configuration gains the bounded unbacked-container-item limit. The wire encoding is unchanged.
## Benchmark
Not run, per explicit maintainer instruction for this robustness task. The implementation preserves positive static/generated loop shape and adds no per-element allocation; focused source/codegen assertions cover removal of budget/cursor work from proven-positive paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5160920210
AI review artifact: Fory-guided full-diff reviewer
- reviewer_label: fresh Fory-guided reviewer
- reviewed_head: e42ab7a3624e8e1bf16c9c6ce379713fe61e1465
- comparison_base: 5971ded1da4a9b308620388546f581e7c90188df
- reviewed_paths: 184
- binary_diff_sha256: 616f28f114b83d7ac965ad384e208aa52e049f02cd62b4759d6c0e4fab24eb56
- result: No actionable findings.
- reviewer_constraints: read-only; no edits, builds, tests, benchmarks, or child-agent delegation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5160967033
Final AI review artifact: independent general reviewer
- reviewer_label: fresh independent general reviewer
- reviewed_head: 855c25280109e55ccb4ea09d8111f42ab45e1113
- comparison_base: 5971ded1da4a9b308620388546f581e7c90188df
- reviewed_paths: 184
- binary_diff_sha256: bbf24010bb4d59ab9ec8f8d33a20eeffd49b57892145ca67606b429bedb6c5ce
- result: No actionable findings.
- reviewer_constraints: read-only; no Fory-specific review checklist; no edits, builds, tests, benchmarks, or child-agent delegation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5161180429
Final Fory-guided full-diff review artifact
- Head: `b71f7f18cf610f25204aa8441dd8c49246b952e9`
- Base: `5971ded1da4a9b308620388546f581e7c90188df`
- Boundary: 186 paths, binary diff SHA-256 `25933faf4082c67b8f63cac4e2764a460168d3aa184dae3b6897b8d5e3c97b8b`
- Reviewer: one fresh read-only Fory-guided reviewer using repository guidance and security/design documents
- Delegation: explicitly prohibited; reviewer created no child agents
- Result: `No actionable findings.`
- CI at reviewed boundary: 87 checks passed, including Windows C++/MSVC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5161180544
Final independent general full-diff review artifact
- Head: `b71f7f18cf610f25204aa8441dd8c49246b952e9`
- Base: `5971ded1da4a9b308620388546f581e7c90188df`
- Boundary: 186 paths, binary diff SHA-256 `25933faf4082c67b8f63cac4e2764a460168d3aa184dae3b6897b8d5e3c97b8b`
- Reviewer: one fresh read-only independent general reviewer; Fory-specific review checklist was explicitly excluded
- Delegation: explicitly prohibited; reviewer created no child agents
- Result: `No actionable findings.`
- CI at reviewed boundary: 87 checks passed, including Windows C++/MSVC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5160920308
AI review artifact: independent general reviewer
- reviewer_label: fresh independent general reviewer
- reviewed_head: e42ab7a3624e8e1bf16c9c6ce379713fe61e1465
- comparison_base: 5971ded1da4a9b308620388546f581e7c90188df
- reviewed_paths: 184
- binary_diff_sha256: 616f28f114b83d7ac965ad384e208aa52e049f02cd62b4759d6c0e4fab24eb56
- result: No actionable findings.
- reviewer_constraints: read-only; no Fory-specific review checklist; no edits, builds, tests, benchmarks, or child-agent delegation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization work [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang commented on PR #3910:
URL: https://github.com/apache/fory/pull/3910#issuecomment-5160966935
Final AI review artifact: Fory-guided full-diff reviewer
- reviewer_label: fresh Fory-guided reviewer
- reviewed_head: 855c25280109e55ccb4ea09d8111f42ab45e1113
- comparison_base: 5971ded1da4a9b308620388546f581e7c90188df
- reviewed_paths: 184
- binary_diff_sha256: bbf24010bb4d59ab9ec8f8d33a20eeffd49b57892145ca67606b429bedb6c5ce
- result: No actionable findings.
- reviewer_constraints: read-only; no edits, builds, tests, benchmarks, or child-agent delegation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat: bound unbacked container deserialization [fory]
Posted by "chaokunyang (via GitHub)" <gi...@apache.org>.
chaokunyang merged PR #3910:
URL: https://github.com/apache/fory/pull/3910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org