[jira] [Commented] (HIVE-21418) Reset aggregate column stat cache before each q test

["Miklos Gergely (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788682#comment-16788682 ] 

Miklos Gergely commented on HIVE-21418:
---------------------------------------

[~thejas] as we've discussed the AggregateStatsCache may return only partial results because of previously generated data. This leads to potential errors in the q tests, as it is not predictable what will be in the cache when a given q test is executed, as it may be grouped with different other q tests from run to run. In my opinion the safest solution for this problem is to clear the cache before every q test run. This would prevent any issues like this in the future, when some issue with the cache could cause strange errors for totally unrelated modifications. I've attached a patch for this.

However according to [~kgyrtkirk] we should rather make sure that the cache is never returning partial result, and thus it wouldn't cause an issue. There is a Jira tracking this issue at HIVE-20585. Please chime in, and let us know what you think.

cc [~ashutoshc]

> Reset aggregate column stat cache before each q test
> ----------------------------------------------------
>
>                 Key: HIVE-21418
>                 URL: https://issues.apache.org/jira/browse/HIVE-21418
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Miklos Gergely
>            Assignee: Miklos Gergely
>            Priority: Major
>         Attachments: HIVE-21418.01.patch
>
>
> Aggregate column stat cache may store statistics for common tables in the q tests like src, src1, srcpart etc, which may affect wether the aggregate stats received would be complete or partial.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)