You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "LuciferYang (via GitHub)" <gi...@apache.org> on 2026/06/23 05:59:25 UTC
[PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
LuciferYang opened a new pull request, #56691:
URL: https://github.com/apache/spark/pull/56691
### What changes were proposed in this pull request?
This PR reuses a single shared `JsonFactory` in the variant module instead of allocating a new one on every call. `Variant.escapeJson` and `VariantBuilder.parseJson` each created a `new JsonFactory()` per invocation; both now reference a `private static final` instance.
### Why are the changes needed?
`JsonFactory` is thread-safe and designed to be reused. `escapeJson` runs once per string value and object key while serializing a variant, and `parseJson` runs once per input JSON value (e.g. per row in `parse_json`), so creating a fresh factory on each call adds avoidable allocation (including its symbol tables) on these hot paths. Sharing one instance removes that overhead.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing variant tests cover both the parsing and serialization paths. There is no behavior change: both factories use the default configuration and are never mutated, so a shared instance produces identical results.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on code in PR #56691:
URL: https://github.com/apache/spark/pull/56691#discussion_r3461428265
##########
common/variant/src/main/java/org/apache/spark/types/variant/VariantBuilder.java:
##########
@@ -42,6 +42,11 @@
* Build variant value and metadata by parsing JSON values.
*/
public class VariantBuilder {
+ // Jackson's JsonFactory is thread-safe and intended to be reused. parseJson is invoked once per
+ // input JSON value (e.g. per row in parse_json), so sharing a single factory avoids allocating
+ // one (and its symbol tables) on every call.
+ private static final JsonFactory JSON_FACTORY = new JsonFactory();
Review Comment:
Nit: two separate JSON_FACTORY statics encode the same default config in Variant.java and VariantBuilder.java. A single shared constant (e.g. on VariantUtil) would avoid future config drift.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b closed pull request #56691: [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module
URL: https://github.com/apache/spark/pull/56691
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #56691:
URL: https://github.com/apache/spark/pull/56691#issuecomment-4794123878
Merging to master/4.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #56691:
URL: https://github.com/apache/spark/pull/56691#issuecomment-4795453342
Thank you @uros-b
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57611][SQL] Reuse JsonFactory instances in the variant module [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #56691:
URL: https://github.com/apache/spark/pull/56691#issuecomment-4786118552
Good call - moved it to a single `JSON_FACTORY` on `VariantUtil` and dropped the two per-class copies (`97dc876`). `Variant` and `VariantBuilder` already static-import `VariantUtil`, so `escapeJson` and `parseJson` keep using it as-is - nothing left to drift now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org