[jira] [Created] (SPARK-57626) Extend shared get_json_object parsing to nested named paths

["Chao Sun (Jira)" <ji...@apache.org>]

Chao Sun created SPARK-57626:
--------------------------------

             Summary: Extend shared get_json_object parsing to nested named paths
                 Key: SPARK-57626
                 URL: https://issues.apache.org/jira/browse/SPARK-57626
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Chao Sun
            Assignee: Chao Sun


SPARK-47670 introduced opt-in shared parsing for repeated get_json_object calls over the same JSON input. Its implementation intentionally shares only simple top-level fields, so repeated literal nested object paths still parse the input independently.

For example:

{code:sql}
SELECT
  get_json_object(json, '$.payload.user.id') AS user_id,
  get_json_object(json, '$.payload.user.name') AS user_name,
  get_json_object(json, '$.payload.request_id') AS request_id
FROM events
{code}

With spark.sql.optimizer.getJsonObjectSharedParsing.enabled=true, these prefix-free named paths should be extracted in one streaming scan without requiring any query changes.

The follow-up should preserve the existing get_json_object behavior for malformed input, duplicate keys, nulls, and rendering failures. An ancestor and its descendant must not share the same parse, because each requested path needs independent legacy semantics. Dynamic paths, wildcards, array subscripts, and excessively deep paths should continue using the existing evaluation.

This is distinct from SPARK-53764, which collapses nested get_json_object function calls rather than sharing sibling paths over the same input.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org