You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by "Mr-LiuXu (via GitHub)" <gi...@apache.org> on 2025/03/04 08:01:28 UTC
[PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Mr-LiuXu opened a new pull request, #5673:
URL: https://github.com/apache/hive/pull/5673
**Fixed the error in parsing the element of the array when the path of the array is not specified in the json data using the **get_json_object** function.**
For example:
json='{"respBody":{"data":[{"score":60},{"score":90}]}}'
The first scenario:
**get_json_object**(json,'$.respBody.data[0].score')
The return result is: 60, which is correct;
The second scenario:
**get_json_object**(json,'$.respBody.data.score[0]')
The return result is also: 60, which is incorrect; RespBody.data is an array below, but it does not specify which element to retrieve from the data array. Instead, it specifies the first element of the array after the score. In this case, a specific value should not be returned as it can cause ambiguity. According to strict mode, it should be returned as null.
### Suggested solution:
The **get_json_object** (json, path, [**arrayStrictMode**]) function adds an optional field, **arrayStrictMode**, which is not passed in and defaults to false, following the previous pattern parsing; If true, follow strict mode. If the json array does not specify which element to retrieve, return empty.
For example:
**get_json_object**(json,'$.respBody.data.score[0]',true)
The return result is: null。
When **get_json_object**(json,'$.respBody.data[0].score', true)
The return result is: 60.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "okumin (via GitHub)" <gi...@apache.org>.
okumin commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2735669490
I also think the current Hive's behavior is weird. Should we expect the current implementation to include a bug? Or can we have a valid reason to justify the current behavior? I guess Hive violates RFC.
I've not looked into the evaluation logic, but it is reasonable for me to fix the behavior. I'm still wondering if we should add a flag or just bug-fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5673: Fix the issue of parsing json arrays using the get_json_object function
URL: https://github.com/apache/hive/pull/5673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2698218396
## [](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5673) **Quality Gate passed**
Issues
 [4 New issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5673&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5673&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive&pullRequest=5673&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5673&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_hive&pullRequest=5673&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_hive&pullRequest=5673)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "Aggarwal-Raghav (via GitHub)" <gi...@apache.org>.
Aggarwal-Raghav commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2739195093
@okumin, if we change the current implementation then it will be a breaking change for user queries who were exploiting this wrong behaviour since previous version of hive. If we change the current implementation then it has to be documented and should go in RELEASE NOTES _(if hive is maintaining)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "Aggarwal-Raghav (via GitHub)" <gi...@apache.org>.
Aggarwal-Raghav commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2702940036
@Mr-LiuXu, thanks for the PR.
I checked the behaviour of get_json_object and it is treating the "score" value as a array i.e [60, 90].
<img width="793" alt="Screenshot 2025-03-06 at 11 53 15 AM" src="https://github.com/user-attachments/assets/25cb005f-b19d-437e-a312-8acab4790360" />
**In spark-sql also, get_json_object for "$.respBody.data.score" returns NULL.**
I also checked the behaviour in python and [jq](https://jqlang.org/) and both of them are throwing error.
<img width="613" alt="Screenshot 2025-03-06 at 11 54 28 AM" src="https://github.com/user-attachments/assets/03acec8f-5387-4364-b587-bf66d704b4fe" />
<img width="712" alt="python behaviour" src="https://github.com/user-attachments/assets/e734a6c2-e899-41f7-9ac0-a855a115fb57" />
IMO, we can keep the behaviour inline with python, jq and spark. The Patch in this PR will help in this and won't break backward compatibility as well.
CC @okumin , can you provide your insights here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2892574632
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "deniskuzZ (via GitHub)" <gi...@apache.org>.
deniskuzZ commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2955822244
pr was closed as stale. do we want to pursue the fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #5673: Fix the issue of parsing json arrays using the get_json_object function
URL: https://github.com/apache/hive/pull/5673
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-3169572544
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
Re: [PR] Fix the issue of parsing json arrays using the get_json_object function [hive]
Posted by "Mr-LiuXu (via GitHub)" <gi...@apache.org>.
Mr-LiuXu commented on PR #5673:
URL: https://github.com/apache/hive/pull/5673#issuecomment-2713641282
@Aggarwal-Raghav Thank you for your reply. Is this repair method acceptable to me? If you have any questions or better suggestions, please reply to me and I will make further modifications. thank
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org