You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Phabricator (Updated) (JIRA)" <ji...@apache.org> on 2011/11/28 22:03:41 UTC
[jira] [Updated] (HIVE-2607) Add caching to json_tuple

     [ https://issues.apache.org/jira/browse/HIVE-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Phabricator updated HIVE-2607:
------------------------------

    Attachment: HIVE-2607.D543.1.patch

kevinwilfong requested code review of "HIVE-2607 [jira] Add caching to json_tuple".
Reviewers: JIRA

  I added an object to serve as a cache to the implementation of json_tuple.  It is copied from get_json_object, I wasn't sure how people would feel about introducinc a dependency between the two functions.

  Whenever a string is not found in the MRU cache, it is parsed into a JSON object, and added to the cache.

TEST PLAN
  I had a specific case, where a query which should have performed better in terms of CPU usage using json_tuple rather than get_json_object, but it was actually performing much worse because the number of unique JSON objects was much smaller than the number of rows.  After this change json_tuple performed significantly better than get_json_object.  I did not see a significant performance degradation in other cases.

  I also ran the unit tests.

REVISION DETAIL
  https://reviews.facebook.net/D543

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFJSONTuple.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/1197/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.

                
> Add caching to json_tuple
> -------------------------
>
>                 Key: HIVE-2607
>                 URL: https://issues.apache.org/jira/browse/HIVE-2607
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-2607.D543.1.patch
>
>
> get_json_object uses a variety of caches to improve its performance.  json_tuple could benefit from having a similar cache from JSON string to JSONObject.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira