You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by "dstepanov (via GitHub)" <gi...@apache.org> on 2026/10/14 15:41:46 UTC
[PR] feat(java): add token-level readUtf8/writeUtf8 operations to ForyJson [fory]
dstepanov opened a new pull request, #4180:
URL: https://github.com/apache/fory/pull/4180
## Why?
[Micronaut](https://micronaut.io) is a JVM framework built around a compile-time-first approach: dependency injection, AOP, bean introspection and configuration are resolved by annotation processors at build time instead of with runtime reflection, proxies or bytecode generation. This keeps startup fast and memory low, and makes applications work in GraalVM native images without reflection configuration.
[Micronaut Serialization](https://github.com/micronaut-projects/micronaut-serialization) applies the same idea to JSON. (De)serializers are generated at compile time, and a small streaming backend does the tokenizing and writing; today that backend is usually jackson-core.
Fory JSON's `Utf8JsonReader` / `Utf8JsonWriter` are a very good fit for that backend role. We built a Micronaut Serialization backend on top of them that keeps Micronaut's generated (de)serializers and uses none of Fory's codecs. On Micronaut's user-bean benchmark it is clearly faster than the same generated code on jackson-core; numbers are below. It passes Micronaut Serialization's full TCK on the JVM and in a GraalVM native image.
The obstacle is that there is no supported way to obtain a reader or writer outside `ForyJson`'s object mapping. Creating one needs a `JsonConfig` (no public constructor; `ForyJson.config()` is package-private) and a `JsonTypeResolver`. Our prototype had to put a helper class into the `org.apache.fory.json` package (a split package that breaks on the module path) and create its own reader instances outside `ForyJson`'s fixed pool. As the comment in the `ForyJson` constructor explains, that pool is intentionally the only owner of execution state, so that field-name caches stay bounded.
This PR adds the smallest supported entry point for that use case and changes nothing else. It is a first step for #4164. A standalone reader/writer artifact can be discussed separately there.
## What does this PR do?
- Adds `ForyJson.readUtf8(byte[] bytes, int offset, int length, Utf8ReadOperation<R, E> operation)`:
- lends the pooled `Utf8JsonReader` for the byte range to the operation;
- rejects trailing non-whitespace content afterwards;
- returns the operation's result.
- Adds `ForyJson.writeUtf8(Utf8WriteOperation<E> operation)`, which lends the pooled `Utf8JsonWriter` and returns the written bytes as a detached array.
- Adds two small functional interfaces, `Utf8ReadOperation<R, E extends Throwable>` and `Utf8WriteOperation<E extends Throwable>`. Checked exceptions thrown by the caller's (de)serializers, such as `IOException`, propagate unchanged without wrapping.
- Both methods reuse the existing root-operation structure: `acquire()` / `release()` of the fixed pool, `lockJIT()` / `unlockJIT()`, and the same buffer handoff and cleanup as `fromJson(byte[], int, int, …)` and `toJsonBytes(…)`. Field-name caches therefore stay bounded, and the readers and writers never escape the pool.
- Tests in `ForyJsonTokenOperationTest`:
- round trip through the lent reader and writer;
- byte ranges;
- trailing-content rejection;
- checked and runtime exception propagation;
- state release after failures, using a single-slot pool that would otherwise block;
- pool reuse.
- Docs: a "Token-level reading and writing" section in `docs/json/integration.md`. It includes a note that external readers should call `enterDepth`/`exitDepth` and `reserveGraphMemory`, as custom codecs do, for the depth and graph-memory limits to apply.
No existing code path is modified, and no existing public API changes.
**Validation from the consumer side:** we rebased the Micronaut Serialization backend onto exactly this API, using a local `1.8.0-SNAPSHOT` build of this branch. It no longer needs the split-package helper or its own reader/writer pool. Micronaut's Fory module tests (1226) and the serialization TCK suite pass unchanged on top of `readUtf8` / `writeUtf8`.
## Related issues
- #4164
## AI Contribution Checklist
- [ ] Substantial AI assistance was used in this PR: `yes`
- [ ] If `yes`, I included a completed [AI Contribution Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs) in this PR description and the required `AI Usage Disclosure`.
- [ ] If `yes`, my PR description includes the required `ai_review` summary and evidence of the final clean AI review results from both fresh reviewers described in `AI_POLICY.md`, on the current HEAD.
```text
AI Usage Disclosure
- substantial_ai_assistance: yes
- scope: code drafting, tests, docs, design discussion
- affected_files_or_subsystems: java/fory-json (ForyJson, Utf8ReadOperation, Utf8WriteOperation, ForyJsonTokenOperationTest), docs/json/integration.md
- ai_review: two fresh AI review sessions on HEAD 00d467e71: (1) Fory-guided reviewer using AGENTS.md and .agents/ci-and-pr.md; its first pass raised one docs finding (external readers must track depth/graph memory), fixed in 00d467e71; second pass: "No further actionable findings". (2) Independent general reviewer in a separate clean session, not pointed at .agents/ci-and-pr.md: "There are no further actionable findings."
- ai_review_artifacts: review session ids 01a13b0c-e3b9-7d12-8bf7-d10539725317 (Fory-guided, final pass) and 01a13b10-54cf-7ca1-8a2e-9bf72c32b8d1 (independent); summaries quoted above
- human_verification: `mvn -pl fory-json clean install` (JDK 25.0.2): 1289 tests pass, checkstyle, spotless, javadoc, JDK25 multi-release jar verifier and GraalVM feature jar verifier pass; Micronaut Serialization backend rebased on this API: 1226 module tests + serialization TCK suite pass
- performance_verification: no existing code path is modified, so ForyJson performance is unchanged; consumer-side benchmark below
- provenance_license_confirmation: Apache-2.0-compatible provenance confirmed; no third-party code introduced
```
## Does this PR introduce any user-facing change?
- [x] Does this PR introduce any public API change? (additive only: `ForyJson.readUtf8`, `ForyJson.writeUtf8`, `Utf8ReadOperation`, `Utf8WriteOperation`)
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
This PR does not modify existing code paths, so existing `ForyJson` performance is unaffected.
For context, these are consumer-side results from Micronaut Serialization's `UserBeanSerdeBenchmark`-style model (users with nested friends, tags and scalars). They compare the same compile-time-generated Micronaut (de)serializers running on jackson-core and on Fory's `Utf8JsonReader` / `Utf8JsonWriter` (JDK 25, 2 forks, 3×2s warmup, 5×2s measurement):
| Stack | serialize (ops/µs, higher is better) | deserialize (µs/op, lower is better) |
|---|---:|---:|
| Micronaut generated serdes on jackson-core | 0.525 | 3.047 |
| Micronaut generated serdes on Fory reader/writer | **1.257** (2.4×) | **2.344** (−23%) |
| ForyJson's own generated codecs (reference) | 1.890 | 0.933 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(java): add token-level readUtf8/writeUtf8 operations to ForyJson [fory]
Posted by "dstepanov (via GitHub)" <gi...@apache.org>.
dstepanov commented on PR #4180:
URL: https://github.com/apache/fory/pull/4180#issuecomment-6335081193
Failure doesn't look related
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(java): add token-level readUtf8/writeUtf8 operations to ForyJson [fory]
Posted by "dstepanov (via GitHub)" <gi...@apache.org>.
dstepanov commented on PR #4180:
URL: https://github.com/apache/fory/pull/4180#issuecomment-6335096949
@chaokunyang Is it acceptable change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org
Re: [PR] feat(java): add token-level readUtf8/writeUtf8 operations to ForyJson [fory]
Posted by "chaokunyang (via GitHub) via commits" <co...@fory.apache.org>.
chaokunyang merged PR #4180:
URL: https://github.com/apache/fory/pull/4180
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org