You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "davidzollo (via GitHub)" <gi...@apache.org> on 2026/07/01 08:35:44 UTC

[I] [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform [seatunnel]

davidzollo opened a new issue, #11249:
URL: https://github.com/apache/seatunnel/issues/11249

   ## Search before asking
   
   - [x] I searched the existing issues and found related topics such as #11061 and #1917, but they do not cover this exact request.
   
   ## Description
   
   ### Background
   
   The current SQL Transform in SeaTunnel is backed by the existing Zeta SQL engine. It does not provide a built-in JSON path extraction function such as `GET_JSON_OBJECT(json_col, '$.key1')`.
   
   At the moment, SeaTunnel users can only work around this by adding a separate `JsonPath` transform before the SQL transform, then referencing the extracted temporary field in SQL.
   
   That workaround exists, but it makes migration from Hive/Spark-style SQL significantly harder because many existing jobs already use JSON extraction inline inside SQL expressions.
   
   ### Example
   
   A real migration case needs to support SQL like this:
   
   ```sql
   SELECT CASE
            WHEN GET_JSON_OBJECT(PATH, '$.key1') = 'value1' THEN TRIM(key1)
            ELSE TRIM(key1)
          END AS new_col
   FROM xxx
   ```
   
   Today this has to be rewritten into multiple transforms and intermediate columns.
   
   ### Why this is not a duplicate of existing issues
   
   - #11061 proposes a new Calcite Transform with native JSON functions such as `JSON_VALUE` and `JSON_QUERY`.
   - That issue is about a new transform plugin, not the current SQL Transform.
   - This request is specifically about improving the existing SQL Transform so simple JSON extraction can be done inline without forcing users to migrate to another transform type or split one SQL into multiple transforms.
   
   ### Proposal
   
   Support one of the following in the existing SQL Transform:
   
   1. Add a built-in `GET_JSON_OBJECT(json, path)` function for compatibility with existing SQL jobs.
   2. Or add an equivalent built-in JSON scalar extraction function and document the compatibility / migration guidance clearly.
   
   ### Expected behavior
   
   - The function should work inside `SELECT`, `CASE WHEN`, `WHERE`, and other ordinary scalar-expression positions.
   - Missing path / null input behavior should be defined clearly.
   - Invalid JSON or invalid JSON path behavior should be documented.
   
   ### Current workaround
   
   Use the standalone `JsonPath` transform first, then reference the extracted field from SQL. This works, but it adds extra configuration and breaks direct SQL portability.
   
   ### Acceptance criteria
   
   - Support JSON path extraction in the current SQL Transform.
   - Cover typical inline usages such as `SELECT`, `CASE WHEN`, and filter conditions.
   - Add unit tests for normal path extraction, null input, missing path, and invalid JSON/path.
   - Update docs with examples and limitations.
   
   ## Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ## Code of Conduct
   
   - [x] I agree to follow this project's Code of Conduct.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo closed issue #11249: [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform
URL: https://github.com/apache/seatunnel/issues/11249


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform [seatunnel]

Posted by "MohamedFazil1406 (via GitHub)" <gi...@apache.org>.
MohamedFazil1406 commented on issue #11249:
URL: https://github.com/apache/seatunnel/issues/11249#issuecomment-4853932137

   Hi i would like to contribute in this project


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform [seatunnel]

Posted by "julianbradford19-png (via GitHub)" <gi...@apache.org>.
julianbradford19-png commented on issue #11249:
URL: https://github.com/apache/seatunnel/issues/11249#issuecomment-4858210421

   Thank you for your interest in contributing to SeaTunnel!
   
   Since David Zollo also expressed interest in submitting a PR for this
   feature, it would be best to coordinate with them on the implementation
   details. You can start by reviewing the proposal for adding GET_JSON_OBJECT
   to the SQL transform and perhaps discuss how to divide the work, such as
   handling the scalar-expression logic or writing the unit tests for null
   inputs and invalid JSON paths.
   
   On Wed, Jul 1, 2026, 6:28 AM Mohamed Fazil ***@***.***> wrote:
   
   > *MohamedFazil1406* left a comment (apache/seatunnel#11249)
   > <https://github.com/apache/seatunnel/issues/11249#issuecomment-4853932137>
   >
   > Hi i would like to contribute in this project
   >
   > —
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/seatunnel/issues/11249?email_source=notifications&email_token=B74KUGZELJHALUZVCXMN2VL5CTYVDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBVGM4TGMRRGM32M4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-4853932137>,
   > or unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/B74KUG2FGE6UIA4LNY27FI35CTYVDAVCNFSNUABEKJSXA33TNF2G64TZHM4TSNBRGIZTAOB3JFZXG5LFHM2DOOBTHA3TQOJZG2QXMAQ>
   > .
   > Triage notifications, keep track of coding agent tasks and review pull
   > requests on the go with GitHub Mobile for iOS
   > <https://github.com/notifications/mobile/ios/B74KUGYKE25KB3BKUUXVKJD5CTYVDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBVGM4TGMRRGM32M4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSVGM33PORSXEX3JN5ZQ>
   > and Android
   > <https://github.com/notifications/mobile/android/B74KUG4646OGKU4YUIV5QYD5CTYVDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBVGM4TGMRRGM32M4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>.
   > Download it today!
   > You are receiving this because you are subscribed to this thread.Message
   > ID: ***@***.***>
   >
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature][Transform-V2] Support GET_JSON_OBJECT in SQL transform [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on issue #11249:
URL: https://github.com/apache/seatunnel/issues/11249#issuecomment-4861292306

   Thanks for volunteering to help on this.
   
   This still looks like a reasonable enhancement request, but I would keep the first implementation boundary quite tight: support inline JSON scalar extraction in the current SQL transform path, and make the null / missing-path / invalid-JSON behavior explicit in tests.
   
   Since @MohamedFazil1406 is already assigned here, I would suggest treating that as the single active owner for the first PR so we do not end up with overlapping implementations. If you would still like to help, coordinating on test coverage, docs, or a follow-up slice would be a good path.
   
   For the initial PR, the most important things are:
   1. keep the scope limited to the current SQL transform rather than mixing in a broader SQL-engine redesign
   2. cover ordinary scalar-expression positions like `SELECT`, `CASE WHEN`, and filters
   3. lock down behavior for null input, missing path, and invalid JSON/path with unit tests
   
   That should give us a much easier review path than trying to solve every JSON-SQL compatibility case at once.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org