[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively

["Phabricator (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Phabricator updated HIVE-2758:
------------------------------

    Attachment: HIVE-2758.D1491.1.patch

kevinwilfong requested code review of "HIVE-2758 [jira] Metastore is caching too aggressively".
Reviewers: JIRA

  https://issues.apache.org/jira/browse/HIVE-2758

  In order to prevent inconsistencies between Hive clients and Hive metastores due to over-aggressive caching, I turned off level 2 caching in datanucleus.  I tried adjusting it to weak caching first, but that did not fix the issue.  I think level 1 caching is the most caching we can have without seeing these issues.

  The metastore is caching values, like table names and locations too aggressively, leading to inconsistencies across Hive clients and metastore Thrift servers.

  For example, open two Hive clients, in each call
  DESCRIBE FORMATTED table_foo;

  Then in one of those clients, execute
  ALTER TABLE table_foo RENAME TO table_bar;

  Then in both clients call
  DESCRIBE FORMATTED table_bar;

  In the client that executed the alter command, the location is correct, however, in the other Hive client, it will still show the original location of table_foo.

  A similar experiment can be done using metastore Thrift servers, substituting get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME TO.

  On the Thrift server you can see that the one which did not execute the alter command, not only returns the wrong location, despite calling get_table('table_bar') it will return a table that still has the name table_foo.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D1491

AFFECTED FILES
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/3105/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.

                
> Metastore is caching too aggressively
> -------------------------------------
>
>                 Key: HIVE-2758
>                 URL: https://issues.apache.org/jira/browse/HIVE-2758
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Kevin Wilfong
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-2758.D1491.1.patch
>
>
> The metastore is caching values, like table names and locations too aggressively, leading to inconsistencies across Hive clients and metastore Thrift servers.
> For example, open two Hive clients, in each call
> DESCRIBE FORMATTED table_foo;
> Then in one of those clients, execute
> ALTER TABLE table_foo RENAME TO table_bar;
> Then in both clients call
> DESCRIBE FORMATTED table_bar;
> In the client that executed the alter command, the location is correct, however, in the other Hive client, it will still show the original location of table_foo.
> A similar experiment can be done using metastore Thrift servers, substituting get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME TO.
> On the Thrift server you can see that the one which did not execute the alter command, not only returns the wrong location, despite calling get_table('table_bar') it will return a table that still has the name table_foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira