[jira] [Created] (IMPALA-8406) Failed REFRESH can partially modify table without bumping version number

["Todd Lipcon (JIRA)" <ji...@apache.org>]

Todd Lipcon created IMPALA-8406:
-----------------------------------

             Summary: Failed REFRESH can partially modify table without bumping version number
                 Key: IMPALA-8406
                 URL: https://issues.apache.org/jira/browse/IMPALA-8406
             Project: IMPALA
          Issue Type: Bug
          Components: Catalog
    Affects Versions: Impala 3.2.0
            Reporter: Todd Lipcon


Currently, various incremental operations in the catalogd modify Table objects in place, including REFRESH, which modifies each partition. In this case, if one partition fails to refresh (eg due to incorrect partitions or some other file access problem), other partitions can still be modified, either because they were modified first (in a non-parallel operation) or modified in parallel (for REFRESH).

In this case, the REFRESH operation will throw an Exception back to the user, but in fact it has modified the catalog entry. The version number, however, is not bumped, which breaks some invariants of the catalog that an object doesn't change without changing version numbers.

This also produces some unexpected behavior such as:
- SHOW FILES IN t;
- REFRESH t; -- gets a failure
- SHOW FILES in t; -- see the same result as originally
- ALTER TABLE t SET UNCACHED; -- bumps the version number due to unrelated operation
- SHOW FILES IN t; -- the set of files has changed due to the earlier partially-complete REFRESH





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org