You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fory.apache.org by ch...@apache.org on 2026/08/23 14:10:05 UTC
(fory) branch main updated: feat(java): parse quoted JSON scalar values (#3967)
This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new 0ee14a4f1 feat(java): parse quoted JSON scalar values (#3967)
0ee14a4f1 is described below
commit 0ee14a4f1b55375d3755a5e6c7e8c17cc7fdd8cb
Author: Shawn Yang <sh...@gmail.com>
AuthorDate: Sun Aug 23 22:10:01 2026 +0800
feat(java): parse quoted JSON scalar values (#3967)
## Why?
Financial and other APIs commonly encode boolean and numeric values as
JSON strings. Fory JSON
previously rejected those values for declared scalar targets, so users
had to register custom
codecs even for standard types such as `BigDecimal`.
## What does this PR do?
- Accept quoted ordinary token text for declared boolean, byte, short,
int, long, float, double,
`BigInteger`, and `BigDecimal` targets without a configuration switch.
- Parse quoted values through the same representation-specific token
parsers as native JSON
tokens, without allocating an intermediate `String` or another per-value
carrier.
- Keep natural dynamic JSON typing: a quoted value read as `Object`
remains a `String`; a quoted
numeric value read as `Number` is a `Double`.
- Cover roots, generated and interpreted object codecs, arrays,
collections, maps, Latin1, UTF-8,
UTF-16, arbitrary-precision limits, and malformed quoted input.
- Document the user-visible coercion behavior.
## Related issues
N/A.
## AI Contribution Checklist
- [ ] Substantial AI assistance was used in this PR: `yes` / `no`
- [ ] If `yes`, I included a completed [AI Contribution
Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs)
in this PR description and the required `AI Usage Disclosure`.
- [ ] If `yes`, my PR description includes the required `ai_review`
summary and screenshot evidence or equivalent persisted links of the
final clean AI review results from both fresh reviewers described in
`AI_POLICY.md`, the Fory-guided reviewer and the independent general
reviewer, on the current PR diff or current HEAD after the latest code
changes.
## Does this PR introduce any user-facing change?
Yes. Declared boolean and numeric JSON targets now accept their ordinary
token text inside a JSON
string. Serialization output remains unchanged.
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
Environment: macOS arm64, OpenJDK 25.0.3, JMH 1.37, one fork/thread,
`-Xms2g -Xmx2g`, one-second
iterations, and `-prof gc`.
Unquoted generated primitive POJO parsing against the exact merge-base
artifact:
| Input | Baseline | Current | Change | Allocation |
| --- | ---: | ---: | ---: | ---: |
| Latin1 | 60.159 ns/op | 58.797 ns/op | -2.263% | 40.000 / 40.000 B/op
|
| UTF-8 | 56.840 ns/op | 55.550 ns/op | -2.270% | 40.000 / 40.000 B/op |
| UTF-16 | 131.191 ns/op | 133.706 ns/op | +1.917% | 40.001 / 40.001
B/op |
Quoted and unquoted normalized allocations are equal to JMH counter
precision:
| Case | Unquoted | Quoted |
| --- | ---: | ---: |
| `BigDecimal` | 184.001 B/op | 184.001 B/op |
| `BigInteger` | 264.001 B/op | 264.002 B/op |
| Combined scalar POJO | 592.003 B/op | 592.003 B/op |
Final-head targeted validation:
```text
ENABLE_FORY_DEBUG_OUTPUT=1 mvn -q -pl fory-json -Dtest=JsonScalarTest \
spotless:check checkstyle:check test
```
139 tests passed with no failures, errors, or skips.
---
AGENTS.md | 6 +
docs/json/object-mapping.md | 7 +
.../org/apache/fory/json/reader/JsonReader.java | 219 +++++++++++++++++----
.../apache/fory/json/reader/Latin1JsonReader.java | 66 +++++++
.../apache/fory/json/reader/Utf16JsonReader.java | 66 +++++++
.../apache/fory/json/reader/Utf8JsonReader.java | 66 +++++++
.../java/org/apache/fory/json/JsonScalarTest.java | 136 ++++++++++++-
7 files changed, 522 insertions(+), 44 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 232e55dbf..8329c0401 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -146,6 +146,12 @@ This is the entry point for AI guidance in Apache Fory. Read this file first, th
- When a user corrects a non-obvious invariant, encode it in the nearest source comment before continuing, and also update `AGENTS.md`, `.agents/**`, docs, or specs when the rule is reusable beyond one file. Do not rely only on chat history, task notes, commit messages, or benchmark logs for corrections that protect security, protocol behavior, ownership, naming, or hot-path performance.
- Reject semantic hacks. Do not bypass broken semantics by deleting cases, simplifying callers, adding coercion hooks, or using workaround fallbacks; fix the underlying bug and prove it with focused tests.
- Protect hot paths. Avoid per-call allocations, callback objects, result tuples or records, unnecessary runtime branches, and wrapper-class substitutions in hot codec/runtime paths; prefer conditional imports and allocation-free concrete implementations where they fit the language.
+- Fory JSON declared boolean and numeric scalar targets accept either their native JSON token or
+ the same token text enclosed in quotes without a configuration gate. Keep coercion in the
+ existing reader operation used by root, generated, array, collection, and map paths; dynamic
+ `Object` quoted values remain strings. Quoted scalar common paths must parse directly from reader
+ storage with no intermediate object allocation, reuse the unquoted token parser, and keep larger
+ quoted handling in a separate cold method so native token parsing does not regress.
- Decoder depth and the generic-type stack paired with that depth use root-operation failure cleanup. Nested decoders decrement depth and pop generic types only after successful child reads; do not add nested `try/finally` to restore them after exceptions. The root operation's `finally`/reset must clear both decoder depth and the generic-type stack.
- Keep public APIs minimal. Public APIs must match user ownership and mental model, not internal implementation details; generated flows stay type-owned, while custom serializer registration stays explicit.
- A Fory instance may register types or serializers only before its first root
diff --git a/docs/json/object-mapping.md b/docs/json/object-mapping.md
index 43fef1058..e64b7116b 100644
--- a/docs/json/object-mapping.md
+++ b/docs/json/object-mapping.md
@@ -145,12 +145,19 @@ Non-finite float and double values use the quoted strings `"NaN"`, `"Infinity"`,
`"-Infinity"`. Use explicit `BigInteger` or `BigDecimal` targets when arbitrary precision must be
preserved.
+Declared boolean and numeric targets also accept their ordinary token text in a JSON string, such
+as `"true"`, `"42"`, or `"123.45"`; no builder option is required. This applies to roots, object
+members, arrays, collections, and maps. Fory continues to write native JSON boolean and number
+tokens. An `Object` target retains natural JSON typing, so a quoted value remains a `String`; a
+quoted numeric value read as `Number` uses `Double`.
+
### Built-in representations
These built-in values use the following ordinary JSON shapes:
| Java type | JSON representation |
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
+| Boolean and numeric scalars | Native JSON boolean or number when written; declared targets also read the same token text quoted |
| Enum | Constant name as a string |
| `Date`, `Calendar`, `java.sql.Date`, `Time`, `Timestamp` | Epoch milliseconds as a number |
| `TimeZone` | Time-zone ID as a string |
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
index 3f2f34356..dfd4359f0 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java
@@ -60,6 +60,10 @@ import org.apache.fory.memory.NativeByteOrder;
* {@link BigInteger} or {@link BigDecimal}; raw number text, primitive scans, and skipped values do
* not inherit that resource policy.
*
+ * <p>Declared boolean and numeric scalars accept their native JSON token or the same token text in
+ * quotes without a configuration gate. Quoted common paths consume only the quotes around the
+ * existing token parser, so they retain its allocation and validation behavior.
+ *
* <p>Readers are mutable and confined to one borrowed {@code ForyJson} state. Concrete reset
* methods borrow an input and reset the cursor, depth, and graph-memory budget; {@code clear()}
* detaches that input and clears the root state before the state returns to the pool. A failed
@@ -68,6 +72,7 @@ import org.apache.fory.memory.NativeByteOrder;
*/
public abstract class JsonReader {
private static final int MAX_BIG_NUMBER_LENGTH = 10_000;
+ private static final int INITIAL_BIG_DECIMAL_BUFFER_SIZE = 64;
private static final byte[] EMPTY_BYTES = new byte[0];
static final int MAX_BIG_DECIMAL_SCALE = 10_000;
private static final int COMPACT_DECIMAL_MAX_SCALE = 18;
@@ -263,9 +268,10 @@ public abstract class JsonReader {
private final QuotedTextView quotedTextView = new QuotedTextView(this);
private final Object[] creatorArguments = new Object[1];
- // Primitive floating fallback reuses this exact-boundary workspace. Reader construction is the
- // cold owner so the first precision-sensitive scalar cannot allocate on the numeric hot path.
- private final byte[] decimalBoundaryDigits = new byte[DECIMAL_BOUNDARY_DIGITS];
+ // Keep all reusable numeric arrays behind the original single workspace reference: adding an
+ // inherited reference here shifts concrete readers' representation fields and measurably harms
+ // their native-token hot paths. Reader construction remains the cold allocation owner.
+ private final NumericWorkspace numericWorkspace = new NumericWorkspace();
protected JsonReader(JsonConfig config, JsonTypeResolver typeResolver) {
this.config = config;
@@ -689,6 +695,13 @@ public abstract class JsonReader {
public final boolean readBoolean() {
skipWhitespace();
+ if (position < length() && charAt(position) == '"') {
+ return readQuotedBoolean();
+ }
+ return readBooleanToken();
+ }
+
+ private boolean readBooleanToken() {
if (startsWith("true")) {
position += 4;
return true;
@@ -699,6 +712,13 @@ public abstract class JsonReader {
throw error("Expected boolean");
}
+ private boolean readQuotedBoolean() {
+ beginQuotedScalar();
+ boolean value = readBooleanToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public final String readNumberAsString() {
skipWhitespace();
return readNumberToken();
@@ -709,6 +729,12 @@ public abstract class JsonReader {
}
private String readNumberToken() {
+ int start = position;
+ scanNumberToken();
+ return slice(start, position);
+ }
+
+ private void scanNumberToken() {
int start = position;
if (position < length() && charAt(position) == '-') {
position++;
@@ -728,11 +754,17 @@ public abstract class JsonReader {
if (start == position) {
throw error("Expected number");
}
- return slice(start, position);
}
public final int readInt() {
skipWhitespace();
+ if (position < length() && charAt(position) == '"') {
+ return readQuotedInt();
+ }
+ return readIntToken();
+ }
+
+ private int readIntToken() {
int start = position;
int result = 0;
int limit = -Integer.MAX_VALUE;
@@ -779,8 +811,22 @@ public abstract class JsonReader {
return negative ? result : -result;
}
+ private int readQuotedInt() {
+ beginQuotedScalar();
+ int value = readIntToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public final long readLong() {
skipWhitespace();
+ if (position < length() && charAt(position) == '"') {
+ return readQuotedLong();
+ }
+ return readLongToken();
+ }
+
+ private long readLongToken() {
int start = position;
long result = 0;
long limit = -Long.MAX_VALUE;
@@ -827,6 +873,13 @@ public abstract class JsonReader {
return negative ? result : -result;
}
+ private long readQuotedLong() {
+ beginQuotedScalar();
+ long value = readLongToken();
+ finishQuotedScalar();
+ return value;
+ }
+
/** Reads one canonical unsigned 32-bit JSON integer and returns its raw bits. */
public final int readUnsignedInt() {
long value = readUnsignedLong();
@@ -881,13 +934,71 @@ public abstract class JsonReader {
public BigInteger readBigInteger() {
skipWhitespace();
- int mark = position;
- try {
- return BigInteger.valueOf(readLong());
- } catch (RuntimeException e) {
- position = mark;
- return parseBigInteger(readNumberAsString());
+ if (position < length() && charAt(position) == '"') {
+ return readQuotedBigInteger();
}
+ return readBigIntegerToken();
+ }
+
+ private BigInteger readBigIntegerToken() {
+ int start = position;
+ long result = 0;
+ long limit = -Long.MAX_VALUE;
+ boolean negative = false;
+ if (position < length() && charAt(position) == '-') {
+ negative = true;
+ limit = Long.MIN_VALUE;
+ position++;
+ }
+ if (position >= length()) {
+ throw error("Expected digit");
+ }
+ char ch = charAt(position);
+ if (ch == '0') {
+ position++;
+ rejectLeadingDigit();
+ rejectFractionOrExponent();
+ return BigInteger.ZERO;
+ }
+ if (ch < '1' || ch > '9') {
+ throw error("Expected digit");
+ }
+ long multmin = limit / 10;
+ boolean overflow = false;
+ while (position < length()) {
+ ch = charAt(position);
+ if (ch < '0' || ch > '9') {
+ break;
+ }
+ if (!overflow) {
+ int digit = ch - '0';
+ if (result < multmin) {
+ overflow = true;
+ } else {
+ result *= 10;
+ if (result < limit + digit) {
+ overflow = true;
+ } else {
+ result -= digit;
+ }
+ }
+ }
+ position++;
+ }
+ rejectFractionOrExponent();
+ if (!overflow) {
+ return BigInteger.valueOf(negative ? result : -result);
+ }
+ // Overflow is normal for this arbitrary-precision target. Keeping it out of exception control
+ // flow avoids a transient allocation whose captured stack would differ on the quoted path.
+ return parseBigInteger(slice(start, position));
+ }
+
+ private BigInteger readQuotedBigInteger() {
+ beginQuotedScalar();
+ BigInteger value = readBigIntegerToken();
+ finishQuotedScalar();
+ return value;
}
public char readChar() {
@@ -975,7 +1086,47 @@ public abstract class JsonReader {
protected final BigDecimal readBigDecimalFallback(int start) {
position = start;
- return parseBigDecimal(readNumberAsString());
+ scanNumberToken();
+ int numberLength = position - start;
+ if (numberLength > MAX_BIG_NUMBER_LENGTH) {
+ throwBigNumberLengthExceeded(position);
+ }
+ char[] chars = numericWorkspace.bigDecimalBuffer;
+ if (chars.length < numberLength) {
+ chars = new char[Math.max(numberLength, chars.length << 1)];
+ numericWorkspace.bigDecimalBuffer = chars;
+ }
+ for (int i = 0; i < numberLength; i++) {
+ chars[i] = charAt(start + i);
+ }
+ BigDecimal value;
+ try {
+ value = new BigDecimal(chars, 0, numberLength);
+ } catch (NumberFormatException e) {
+ throw new ForyJsonException("Invalid JSON big decimal at JSON position " + position, e);
+ }
+ int scale = value.scale();
+ if (scale > MAX_BIG_DECIMAL_SCALE || scale < -MAX_BIG_DECIMAL_SCALE) {
+ throwBigDecimalScaleExceeded();
+ }
+ return value;
+ }
+
+ protected final void beginQuotedScalar() {
+ position++;
+ // Concrete token parsers also dispatch an opening quote. Reject another raw quote at this
+ // boundary so malformed nested quoting cannot recursively re-enter the quoted cold path.
+ if (position < length() && charAt(position) == '"') {
+ throw error("Expected quoted scalar value");
+ }
+ }
+
+ protected final void finishQuotedScalar() {
+ if (position < length() && charAt(position) == '"') {
+ position++;
+ return;
+ }
+ throw error("Expected closing quote");
}
protected final BigDecimal readBigDecimalExponentValue(
@@ -1392,23 +1543,6 @@ public abstract class JsonReader {
}
}
- final BigDecimal parseBigDecimal(String number) {
- if (number.length() > MAX_BIG_NUMBER_LENGTH) {
- throwBigNumberLengthExceeded(position);
- }
- BigDecimal value;
- try {
- value = new BigDecimal(number);
- } catch (NumberFormatException e) {
- throw new ForyJsonException("Invalid JSON big decimal at JSON position " + position, e);
- }
- int scale = value.scale();
- if (scale > MAX_BIG_DECIMAL_SCALE || scale < -MAX_BIG_DECIMAL_SCALE) {
- throwBigDecimalScaleExceeded();
- }
- return value;
- }
-
final void throwBigDecimalScaleExceeded() {
throw error("JSON big decimal scale " + MAX_BIG_DECIMAL_SCALE + " exceeded");
}
@@ -1627,9 +1761,6 @@ public abstract class JsonReader {
protected final double readDoubleFallbackValue(int start) {
position = start;
- if (start < length() && charAt(start) == '"') {
- return readNonFiniteDoubleLiteral();
- }
return readDoubleNumberFallback(start);
}
@@ -1806,7 +1937,7 @@ public abstract class JsonReader {
private double correctDoubleToken(boolean negative, double estimate, int start, int end) {
long bits = Double.doubleToRawLongBits(estimate) & ~DOUBLE_SIGN_BIT;
- byte[] boundary = decimalBoundaryDigits;
+ byte[] boundary = numericWorkspace.decimalBoundaryDigits;
// Eighteen retained digits, one correctly rounded multiply/divide, and Math.pow's one-ULP
// contract keep the estimate within this local window. The exact search is a correctness-only
// fallback and is not expected on valid JDK implementations.
@@ -1940,9 +2071,6 @@ public abstract class JsonReader {
protected final float readFloatFallbackValue(int start) {
position = start;
- if (start < length() && charAt(start) == '"') {
- return readNonFiniteFloatLiteral();
- }
return readFloatNumberFallback(start);
}
@@ -2110,7 +2238,7 @@ public abstract class JsonReader {
private float correctFloatToken(boolean negative, float estimate, int start, int end) {
int bits = Float.floatToRawIntBits(estimate);
bits &= ~FLOAT_SIGN_BIT;
- byte[] boundary = decimalBoundaryDigits;
+ byte[] boundary = numericWorkspace.decimalBoundaryDigits;
for (int i = 0; i < 4; i++) {
if (bits == FLOAT_INFINITY_BITS) {
int packed = buildFloatBoundary(FLOAT_MAX_FINITE_BITS, FLOAT_INFINITY_BITS, boundary);
@@ -2481,9 +2609,7 @@ public abstract class JsonReader {
position += 11;
return Double.NEGATIVE_INFINITY;
}
- // Numeric strings are intentionally not coerced; only writer-emitted non-finite tokens
- // are accepted here.
- throw error("Expected finite JSON number or non-finite double string");
+ throw error("Expected quoted double");
}
protected final float readNonFiniteFloatLiteral() {
@@ -2499,9 +2625,13 @@ public abstract class JsonReader {
position += 11;
return Float.NEGATIVE_INFINITY;
}
- // Numeric strings are intentionally not coerced; only writer-emitted non-finite tokens
- // are accepted here.
- throw error("Expected finite JSON number or non-finite float string");
+ throw error("Expected quoted float");
+ }
+
+ protected final boolean isQuotedNonFiniteNumber() {
+ return matchesQuotedAscii("NaN")
+ || matchesQuotedAscii("Infinity")
+ || matchesQuotedAscii("-Infinity");
}
private boolean matchesQuotedAscii(String value) {
@@ -3102,6 +3232,11 @@ public abstract class JsonReader {
protected abstract String slice(int start, int end);
+ private static final class NumericWorkspace {
+ private final byte[] decimalBoundaryDigits = new byte[DECIMAL_BOUNDARY_DIGITS];
+ private char[] bigDecimalBuffer = new char[INITIAL_BIG_DECIMAL_BUFFER_SIZE];
+ }
+
private static final class QuotedTextView implements CharSequence {
private final JsonReader reader;
private byte[] decodedBytes;
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java b/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
index fae818ec2..5a7e9a8b9 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/Latin1JsonReader.java
@@ -512,7 +512,17 @@ public final class Latin1JsonReader extends JsonReader {
return readBooleanToken();
}
+ private boolean readQuotedBooleanValue() {
+ beginQuotedScalar();
+ boolean value = readBooleanToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private boolean readBooleanToken() {
+ if (position < input.length && input[position] == '"') {
+ return readQuotedBooleanValue();
+ }
if (startsWithAscii("true")) {
position += 4;
return true;
@@ -539,6 +549,13 @@ public final class Latin1JsonReader extends JsonReader {
return readIntToken();
}
+ private int readQuotedIntValue() {
+ beginQuotedScalar();
+ int value = readIntToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private int readIntToken() {
byte[] bytes = input;
int offset = position;
@@ -547,6 +564,9 @@ public final class Latin1JsonReader extends JsonReader {
throw error("Expected digit");
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedIntValue();
+ }
if (ch == '-') {
return readNegativeIntToken(offset);
}
@@ -673,6 +693,13 @@ public final class Latin1JsonReader extends JsonReader {
return readLongToken();
}
+ private long readQuotedLongValue() {
+ beginQuotedScalar();
+ long value = readLongToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private long readLongToken() {
byte[] bytes = input;
int offset = position;
@@ -681,6 +708,9 @@ public final class Latin1JsonReader extends JsonReader {
throw error("Expected digit");
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedLongValue();
+ }
if (ch == '-') {
return readNegativeLongToken(offset);
}
@@ -839,6 +869,13 @@ public final class Latin1JsonReader extends JsonReader {
return readBigDecimalToken();
}
+ private BigDecimal readQuotedBigDecimalValue() {
+ beginQuotedScalar();
+ BigDecimal value = readBigDecimalToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public UUID readUuid() {
skipWhitespaceFast();
int mark = position;
@@ -876,6 +913,16 @@ public final class Latin1JsonReader extends JsonReader {
return readDoubleToken();
}
+ private double readQuotedDoubleValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteDoubleLiteral();
+ }
+ beginQuotedScalar();
+ double value = readDoubleToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public float readNextFloatValue() {
if (position < input.length) {
int ch = input[position];
@@ -890,6 +937,16 @@ public final class Latin1JsonReader extends JsonReader {
return readFloatToken();
}
+ private float readQuotedFloatValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteFloatLiteral();
+ }
+ beginQuotedScalar();
+ float value = readFloatToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private BigDecimal readBigDecimalToken() {
byte[] bytes = input;
int offset = position;
@@ -899,6 +956,9 @@ public final class Latin1JsonReader extends JsonReader {
return readBigDecimalFallback(start);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedBigDecimalValue();
+ }
if (ch == '-') {
return readSignedBigDecimalToken(start);
}
@@ -1074,6 +1134,9 @@ public final class Latin1JsonReader extends JsonReader {
return readDoubleFallback(offset);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedDoubleValue();
+ }
if (ch == '-') {
return readSignedDoubleToken(offset);
}
@@ -1088,6 +1151,9 @@ public final class Latin1JsonReader extends JsonReader {
return readFloatFallback(offset);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedFloatValue();
+ }
if (ch == '-') {
return readSignedFloatToken(offset);
}
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
index a487558ed..b7ba21969 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf16JsonReader.java
@@ -561,7 +561,17 @@ public final class Utf16JsonReader extends JsonReader {
return readBooleanToken();
}
+ private boolean readQuotedBooleanValue() {
+ beginQuotedScalar();
+ boolean value = readBooleanToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private boolean readBooleanToken() {
+ if (position < length && charAtFast(position) == '"') {
+ return readQuotedBooleanValue();
+ }
if (startsWithAscii("true")) {
position += 4;
return true;
@@ -588,6 +598,13 @@ public final class Utf16JsonReader extends JsonReader {
return readIntToken();
}
+ private int readQuotedIntValue() {
+ beginQuotedScalar();
+ int value = readIntToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private int readIntToken() {
int offset = position;
int inputLength = length;
@@ -595,6 +612,9 @@ public final class Utf16JsonReader extends JsonReader {
throw error("Expected digit");
}
char ch = charAtFast(offset);
+ if (ch == '"') {
+ return readQuotedIntValue();
+ }
if (ch == '-') {
return readNegativeIntToken(offset);
}
@@ -705,6 +725,13 @@ public final class Utf16JsonReader extends JsonReader {
return readLongToken();
}
+ private long readQuotedLongValue() {
+ beginQuotedScalar();
+ long value = readLongToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private long readLongToken() {
int offset = position;
int inputLength = length;
@@ -712,6 +739,9 @@ public final class Utf16JsonReader extends JsonReader {
throw error("Expected digit");
}
char ch = charAtFast(offset);
+ if (ch == '"') {
+ return readQuotedLongValue();
+ }
if (ch == '-') {
return readNegativeLongToken(offset);
}
@@ -812,6 +842,13 @@ public final class Utf16JsonReader extends JsonReader {
return readBigDecimalToken();
}
+ private BigDecimal readQuotedBigDecimalValue() {
+ beginQuotedScalar();
+ BigDecimal value = readBigDecimalToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private BigDecimal readBigDecimalToken() {
int offset = position;
int start = offset;
@@ -820,6 +857,9 @@ public final class Utf16JsonReader extends JsonReader {
return readBigDecimalFallback(start);
}
char ch = charAtFast(offset);
+ if (ch == '"') {
+ return readQuotedBigDecimalValue();
+ }
if (ch == '-') {
return readSignedBigDecimalToken(start);
}
@@ -962,6 +1002,16 @@ public final class Utf16JsonReader extends JsonReader {
return readDoubleToken();
}
+ private double readQuotedDoubleValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteDoubleLiteral();
+ }
+ beginQuotedScalar();
+ double value = readDoubleToken();
+ finishQuotedScalar();
+ return value;
+ }
+
@Override
public float readFloat() {
skipWhitespaceFast();
@@ -982,6 +1032,16 @@ public final class Utf16JsonReader extends JsonReader {
return readFloatToken();
}
+ private float readQuotedFloatValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteFloatLiteral();
+ }
+ beginQuotedScalar();
+ float value = readFloatToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private double readDoubleToken() {
int offset = position;
int inputLength = length;
@@ -989,6 +1049,9 @@ public final class Utf16JsonReader extends JsonReader {
return readDoubleFallback(offset);
}
char ch = charAtFast(offset);
+ if (ch == '"') {
+ return readQuotedDoubleValue();
+ }
if (ch == '-') {
return readSignedDoubleToken(offset);
}
@@ -1002,6 +1065,9 @@ public final class Utf16JsonReader extends JsonReader {
return readFloatFallback(offset);
}
char ch = charAtFast(offset);
+ if (ch == '"') {
+ return readQuotedFloatValue();
+ }
if (ch == '-') {
return readSignedFloatToken(offset);
}
diff --git a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
index 8415c0e2f..fbfcbe306 100644
--- a/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
+++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/Utf8JsonReader.java
@@ -750,9 +750,19 @@ public final class Utf8JsonReader extends JsonReader {
return readBooleanToken();
}
+ private boolean readQuotedBooleanValue() {
+ beginQuotedScalar();
+ boolean value = readBooleanToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private boolean readBooleanToken() {
byte[] bytes = input;
int offset = position;
+ if (offset < inputLimit && bytes[offset] == '"') {
+ return readQuotedBooleanValue();
+ }
if (offset + 3 < inputLimit
&& bytes[offset] == 't'
&& bytes[offset + 1] == 'r'
@@ -791,6 +801,13 @@ public final class Utf8JsonReader extends JsonReader {
return readIntToken();
}
+ private int readQuotedIntValue() {
+ beginQuotedScalar();
+ int value = readIntToken();
+ finishQuotedScalar();
+ return value;
+ }
+
private int readIntToken() {
byte[] bytes = input;
int offset = position;
@@ -799,6 +816,9 @@ public final class Utf8JsonReader extends JsonReader {
throw error("Expected digit");
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedIntValue();
+ }
if (ch == '-') {
return readNegativeIntToken(offset);
}
@@ -915,11 +935,25 @@ public final class Utf8JsonReader extends JsonReader {
return readLongToken();
}
+ private long readQuotedLongValue() {
+ beginQuotedScalar();
+ long value = readLongToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public BigDecimal readBigDecimal() {
skipWhitespaceFast();
return readBigDecimalToken();
}
+ private BigDecimal readQuotedBigDecimalValue() {
+ beginQuotedScalar();
+ BigDecimal value = readBigDecimalToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public UUID readUuid() {
skipWhitespaceFast();
int mark = position;
@@ -957,6 +991,16 @@ public final class Utf8JsonReader extends JsonReader {
return readDoubleToken();
}
+ private double readQuotedDoubleValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteDoubleLiteral();
+ }
+ beginQuotedScalar();
+ double value = readDoubleToken();
+ finishQuotedScalar();
+ return value;
+ }
+
public float readNextFloatValue() {
if (position < inputLimit) {
int ch = input[position];
@@ -971,6 +1015,16 @@ public final class Utf8JsonReader extends JsonReader {
return readFloatToken();
}
+ private float readQuotedFloatValue() {
+ if (isQuotedNonFiniteNumber()) {
+ return readNonFiniteFloatLiteral();
+ }
+ beginQuotedScalar();
+ float value = readFloatToken();
+ finishQuotedScalar();
+ return value;
+ }
+
// Long parsing deliberately repeats the initial digit checks, zero handling, block scan, and
// short tail used by Int parsing instead of sharing one generic token loop. The widths have
// different safe digit counts, overflow rules, and runtime profiles; a small shared helper lets
@@ -986,6 +1040,9 @@ public final class Utf8JsonReader extends JsonReader {
throw error("Expected digit");
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedLongValue();
+ }
if (ch == '-') {
return readNegativeLongToken(offset);
}
@@ -1207,6 +1264,9 @@ public final class Utf8JsonReader extends JsonReader {
return readBigDecimalFallback(start);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedBigDecimalValue();
+ }
if (ch == '-') {
return readSignedBigDecimalToken(start);
}
@@ -1384,6 +1444,9 @@ public final class Utf8JsonReader extends JsonReader {
return readDoubleFallback(offset);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedDoubleValue();
+ }
if (ch == '-') {
return readSignedDoubleToken(offset);
}
@@ -1398,6 +1461,9 @@ public final class Utf8JsonReader extends JsonReader {
return readFloatFallback(offset);
}
int ch = bytes[offset];
+ if (ch == '"') {
+ return readQuotedFloatValue();
+ }
if (ch == '-') {
return readSignedFloatToken(offset);
}
diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonScalarTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonScalarTest.java
index 16dc0449b..14c422c4b 100644
--- a/java/fory-json/src/test/java/org/apache/fory/json/JsonScalarTest.java
+++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonScalarTest.java
@@ -183,7 +183,7 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertTrue(optional.isPresent());
assertEquals(optional.getAsDouble(), Double.POSITIVE_INFINITY);
- assertThrows(ForyJsonException.class, () -> json.fromJson("\"1.0\"", Double.class));
+ assertEquals(json.fromJson("\"1.0\"", Double.class), Double.valueOf(1.0d));
assertThrows(ForyJsonException.class, () -> json.fromJson("\"nan\"", Float.class));
assertThrows(ForyJsonException.class, () -> json.fromJson("NaN", Double.class));
}
@@ -241,6 +241,17 @@ public class JsonScalarTest extends ForyJsonTestModels {
"{\"bool\":false,\"byteValue\":6,\"charValue\":\"z\",\"doubleValue\":3.5,"
+ "\"floatValue\":2.5,\"intValue\":8,\"longValue\":9,\"shortValue\":7}",
BoxedScalars.class);
+ assertBoxedScalars(value);
+ value =
+ json.fromJson(
+ "{\"bool\":\"false\",\"byteValue\":\"6\",\"charValue\":\"z\","
+ + "\"doubleValue\":\"3.5\",\"floatValue\":\"2.5\",\"intValue\":\"8\","
+ + "\"longValue\":\"9\",\"shortValue\":\"7\"}",
+ BoxedScalars.class);
+ assertBoxedScalars(value);
+ }
+
+ private static void assertBoxedScalars(BoxedScalars value) {
assertEquals(value.bool, Boolean.FALSE);
assertEquals(value.byteValue, Byte.valueOf((byte) 6));
assertEquals(value.charValue, Character.valueOf('z'));
@@ -264,6 +275,18 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertPrimitiveFields(
json.fromJson(values.getBytes(StandardCharsets.UTF_8), PrimitiveFields.class));
+ String quotedValues =
+ "{\"bool\":\"true\",\"byteValue\":\"2\",\"shortValue\":\"3\",\"intValue\":\"4\","
+ + "\"longValue\":\"5\",\"floatValue\":\"1.5\",\"doubleValue\":\"2.5\","
+ + "\"charValue\":\"x\"}";
+ assertPrimitiveFields(json.fromJson(quotedValues, PrimitiveFields.class));
+ assertPrimitiveFields(
+ json.fromJson(
+ quotedValues.replace("}", ",\"text\":\"" + ZH_TEXT + "\"}"), PrimitiveFields.class));
+ assertPrimitiveFields(
+ json.fromJson(quotedValues.getBytes(StandardCharsets.UTF_8), PrimitiveFields.class));
+ assertGeneratedWhenSupported(json, PrimitiveFields.class, codegen);
+
String[] names = {
"bool",
"byteValue",
@@ -1040,6 +1063,17 @@ public class JsonScalarTest extends ForyJsonTestModels {
ForyJson json = newJson();
assertEquals(json.fromJson("7", int.class), Integer.valueOf(7));
assertEquals(json.fromJson("true", boolean.class), Boolean.TRUE);
+ assertEquals(json.fromJson("\"true\"", boolean.class), Boolean.TRUE);
+ assertEquals(json.fromJson("\"2\"", byte.class), Byte.valueOf((byte) 2));
+ assertEquals(json.fromJson("\"3\"", short.class), Short.valueOf((short) 3));
+ assertEquals(json.fromJson("\"4\"", int.class), Integer.valueOf(4));
+ assertEquals(json.fromJson("\"5\"", long.class), Long.valueOf(5));
+ assertEquals(json.fromJson("\"1.5\"", float.class), Float.valueOf(1.5f));
+ assertEquals(
+ json.fromJson("\"2.5\"".getBytes(StandardCharsets.UTF_8), double.class),
+ Double.valueOf(2.5d));
+ assertEquals(json.fromJson("\"123\"", BigInteger.class), BigInteger.valueOf(123));
+ assertEquals(json.fromJson("\"0.100\"", BigDecimal.class), new BigDecimal("0.100"));
assertEquals(json.fromJson("0.100", BigDecimal.class), new BigDecimal("0.100"));
assertEquals(json.fromJson("\"fory\"".getBytes(StandardCharsets.UTF_8), String.class), "fory");
assertEquals(
@@ -1128,6 +1162,54 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertEquals(utf16Reader(duration).readDuration(), expectedDuration);
}
+ @Test(dataProvider = "enableCodegen")
+ public void readQuotedBigNumbers(boolean codegen) {
+ ForyJson json = newJson(codegen);
+ BigInteger integer = new BigInteger("123456789012345678901234567890");
+ BigDecimal decimal = new BigDecimal("12345678901234567890.1234500");
+
+ assertEquals(json.fromJson("\"" + integer + "\"", BigInteger.class), integer);
+ assertEquals(
+ json.fromJson(("\"" + decimal + "\"").getBytes(StandardCharsets.UTF_8), BigDecimal.class),
+ decimal);
+
+ String input = "{\"decimal\":\"" + decimal + "\",\"integer\":\"" + integer + "\"}";
+ Utf8ScalarFields latin1 = json.fromJson(input, Utf8ScalarFields.class);
+ Utf8ScalarFields utf8 =
+ json.fromJson(input.getBytes(StandardCharsets.UTF_8), Utf8ScalarFields.class);
+ assertEquals(latin1.decimal, decimal);
+ assertEquals(latin1.integer, integer);
+ assertEquals(utf8.decimal, decimal);
+ assertEquals(utf8.integer, integer);
+ assertGeneratedWhenSupported(json, Utf8ScalarFields.class, codegen);
+
+ assertQuotedBigIntegerReaders("123456789012345678901234567890");
+ assertQuotedBigDecimalReaders("12345678901234567890.1234500");
+ }
+
+ @Test(dataProvider = "enableCodegen")
+ public void readQuotedScalarContainers(boolean codegen) {
+ ForyJson json = newJson(codegen);
+ assertEquals(
+ json.fromJson("[\"true\",\"false\"]".getBytes(StandardCharsets.UTF_8), boolean[].class),
+ new boolean[] {true, false});
+ assertEquals(json.fromJson("[\"2\",\"3\"]", byte[].class), new byte[] {2, 3});
+ assertEquals(json.fromJson("[\"4\",\"5\"]", short[].class), new short[] {4, 5});
+ assertEquals(json.fromJson("[\"6\",\"7\"]", int[].class), new int[] {6, 7});
+ assertEquals(
+ json.fromJson("[\"8\",\"9\"]".getBytes(StandardCharsets.UTF_8), long[].class),
+ new long[] {8, 9});
+ assertEquals(json.fromJson("[\"1.5\",\"2.5\"]", float[].class), new float[] {1.5f, 2.5f});
+ assertEquals(json.fromJson("[\"3.5\",\"4.5\"]", double[].class), new double[] {3.5d, 4.5d});
+ assertEquals(
+ json.fromJson(
+ "[\"10\",\"11\"]".getBytes(StandardCharsets.UTF_8), new TypeRef<List<Integer>>() {}),
+ Arrays.asList(10, 11));
+ assertEquals(
+ json.fromJson("{\"value\":\"12.5\"}", new TypeRef<Map<String, Double>>() {}).get("value"),
+ Double.valueOf(12.5d));
+ }
+
@Test
public void readQuotedText() {
assertQuotedText("\"fory-json\"", "fory-json", true);
@@ -1330,7 +1412,7 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertThrows(ForyJsonException.class, () -> json.fromJson("01", Number.class));
assertThrows(ForyJsonException.class, () -> json.fromJson("1.", Number.class));
assertThrows(ForyJsonException.class, () -> json.fromJson("\"nan\"", Number.class));
- assertThrows(ForyJsonException.class, () -> json.fromJson("\"1.25\"", Number.class));
+ assertEquals(json.fromJson("\"1.25\"", Number.class), Double.valueOf(1.25d));
assertThrows(
ForyJsonException.class, () -> json.fromJson("\"\\u004e\\u0061\\u004e\"", Number.class));
@@ -1733,6 +1815,10 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertThrows(
ForyJsonException.class,
() -> json.fromJson(repeat('1', BIG_NUMBER_LIMIT + 1), BigInteger.class));
+ assertEquals(json.fromJson("\"" + accepted + "\"", BigInteger.class), new BigInteger(accepted));
+ assertThrows(
+ ForyJsonException.class,
+ () -> json.fromJson("\"" + repeat('1', BIG_NUMBER_LIMIT + 1) + "\"", BigInteger.class));
}
@Test
@@ -1745,6 +1831,10 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertThrows(
ForyJsonException.class,
() -> json.fromJson(repeat('1', BIG_NUMBER_LIMIT + 1), BigDecimal.class));
+ assertEquals(json.fromJson("\"" + accepted + "\"", BigDecimal.class), new BigDecimal(accepted));
+ assertThrows(
+ ForyJsonException.class,
+ () -> json.fromJson("\"" + repeat('1', BIG_NUMBER_LIMIT + 1) + "\"", BigDecimal.class));
String overflowFallback = repeat('9', 20) + "." + repeat('1', BIG_NUMBER_LIMIT + 1);
assertBigDecimalLengthReject(newUtf8Reader(overflowFallback.getBytes(StandardCharsets.UTF_8)));
assertBigDecimalLengthReject(newLatin1Reader(latin1Bytes(overflowFallback)));
@@ -1755,6 +1845,7 @@ public class JsonScalarTest extends ForyJsonTestModels {
public void guardBigDecimalScale() {
ForyJson json = newJson();
assertThrows(ForyJsonException.class, () -> json.fromJson("1e-10001", BigDecimal.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"1e-10001\"", BigDecimal.class));
assertBigDecimalReaders("1e10000");
assertBigDecimalReaders("0.1e10001");
assertBigDecimalReaders("0.1e-9999");
@@ -1788,6 +1879,8 @@ public class JsonScalarTest extends ForyJsonTestModels {
public void rejectInvalidBigNumbers() {
ForyJson json = newJson();
assertThrows(ForyJsonException.class, () -> json.fromJson("1.5", BigInteger.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"1.5\"", BigInteger.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"value\"", BigDecimal.class));
assertThrows(ForyJsonException.class, () -> json.fromJson("1e2147483648", BigDecimal.class));
assertThrows(
ForyJsonException.class,
@@ -1797,6 +1890,29 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertThrows(ForyJsonException.class, () -> utf16Reader("1e2").readBigInteger());
}
+ @Test
+ public void rejectInvalidQuotedScalars() {
+ ForyJson json = newJson();
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"truth\"", boolean.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"01\"", int.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"+1\"", long.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"32768\"", short.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"1x\"", double.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"1.5", float.class));
+
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"true\"\"", boolean.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", int.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", long.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", float.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", double.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", BigInteger.class));
+ assertThrows(ForyJsonException.class, () -> json.fromJson("\"\"1\"\"", BigDecimal.class));
+ assertThrows(
+ ForyJsonException.class,
+ () -> json.fromJson("\"\"1\"\"".getBytes(StandardCharsets.UTF_8), int.class));
+ assertThrows(ForyJsonException.class, () -> utf16Reader("\"\"1\"\"").readIntValue());
+ }
+
@Test
public void readCompactBigDecimalExponents() {
assertBigDecimalReaders("1.25e2");
@@ -3098,6 +3214,14 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertEquals(utf16Reader(token).readBigInteger(), expected);
}
+ private static void assertQuotedBigIntegerReaders(String token) {
+ String quoted = "\"" + token + "\"";
+ BigInteger expected = new BigInteger(token);
+ assertEquals(newUtf8Reader(quoted.getBytes(StandardCharsets.UTF_8)).readBigInteger(), expected);
+ assertEquals(newLatin1Reader(latin1Bytes(quoted)).readBigInteger(), expected);
+ assertEquals(utf16Reader(quoted).readBigInteger(), expected);
+ }
+
private static void assertBigDecimalReaders(String token) {
BigDecimal expected = new BigDecimal(token);
assertEquals(newUtf8Reader(token.getBytes(StandardCharsets.UTF_8)).readBigDecimal(), expected);
@@ -3105,6 +3229,14 @@ public class JsonScalarTest extends ForyJsonTestModels {
assertEquals(utf16Reader(token).readBigDecimal(), expected);
}
+ private static void assertQuotedBigDecimalReaders(String token) {
+ String quoted = "\"" + token + "\"";
+ BigDecimal expected = new BigDecimal(token);
+ assertEquals(newUtf8Reader(quoted.getBytes(StandardCharsets.UTF_8)).readBigDecimal(), expected);
+ assertEquals(newLatin1Reader(latin1Bytes(quoted)).readBigDecimal(), expected);
+ assertEquals(utf16Reader(quoted).readBigDecimal(), expected);
+ }
+
private static void assertSubtypeRejected(Runnable action, Class<?> type) {
ForyJsonException error = expectThrows(ForyJsonException.class, action::run);
assertTrue(error.getMessage().contains(type.getName()));
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@fory.apache.org
For additional commands, e-mail: commits-help@fory.apache.org