You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jason Dere <jd...@hortonworks.com> on 2018/03/29 18:56:14 UTC
Review Request 66368: HIVE-18609: Results cache invalidation based on
table updates
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66368/
-----------------------------------------------------------
Review request for hive, Gopal V and Jesús Camacho Rodríguez.
Bugs: HIVE-18609
https://issues.apache.org/jira/browse/HIVE-18609
Repository: hive-git
Description
-------
- Save ValidTxnWriteIdList when saving query to the results cache.
- Compare the write ID list for each transactional table during results cache lookup.
- Add configuration to determine if queries with non-transactional tables should be cached.
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194
itests/src/test/resources/testconfiguration.properties 0d6aaae535
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 667f436a89
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java 4fa1044232
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 1828f0a531
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 26f20f2e05
ql/src/test/queries/clientpositive/results_cache_1.q 4aea60e1e5
ql/src/test/queries/clientpositive/results_cache_2.q 96a90925f6
ql/src/test/queries/clientpositive/results_cache_capacity.q 9f54577009
ql/src/test/queries/clientpositive/results_cache_empty_result.q 621367141e
ql/src/test/queries/clientpositive/results_cache_invalidation.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q 60ffe96a04
ql/src/test/queries/clientpositive/results_cache_quoted_identifiers.q 4802f43ba9
ql/src/test/queries/clientpositive/results_cache_temptable.q 9e0de765cb
ql/src/test/queries/clientpositive/results_cache_transactional.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q b4fcdd57eb
ql/src/test/results/clientpositive/llap/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_transactional.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_transactional.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66368/diff/1/
Testing
-------
qtests added.
Thanks,
Jason Dere
Re: Review Request 66368: HIVE-18609: Results cache invalidation
based on table updates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66368/
-----------------------------------------------------------
(Updated April 12, 2018, 8:09 p.m.)
Review request for hive, Gopal V and Jesús Camacho Rodríguez.
Changes
-------
- When removing invalid entries during lookup, make sure we have exited read lock section.
- Add results_cache_transactional.q to testconfiguration.properties
Bugs: HIVE-18609
https://issues.apache.org/jira/browse/HIVE-18609
Repository: hive-git
Description
-------
- Save ValidTxnWriteIdList when saving query to the results cache.
- Compare the write ID list for each transactional table during results cache lookup.
- Add configuration to determine if queries with non-transactional tables should be cached.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e540d023bd
itests/src/test/resources/testconfiguration.properties 48d62a8bf9
ql/src/java/org/apache/hadoop/hive/ql/Driver.java a88453c978
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java b1a3646624
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 44a7496136
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 10982ddbd1
ql/src/test/queries/clientpositive/results_cache_1.q 4aea60e1e5
ql/src/test/queries/clientpositive/results_cache_2.q 96a90925f6
ql/src/test/queries/clientpositive/results_cache_capacity.q 9f54577009
ql/src/test/queries/clientpositive/results_cache_empty_result.q 621367141e
ql/src/test/queries/clientpositive/results_cache_invalidation.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q 60ffe96a04
ql/src/test/queries/clientpositive/results_cache_quoted_identifiers.q 4802f43ba9
ql/src/test/queries/clientpositive/results_cache_temptable.q 9e0de765cb
ql/src/test/queries/clientpositive/results_cache_transactional.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q b4fcdd57eb
ql/src/test/results/clientpositive/llap/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_transactional.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_transactional.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66368/diff/4/
Changes: https://reviews.apache.org/r/66368/diff/3-4/
Testing
-------
qtests added.
Thanks,
Jason Dere
Re: Review Request 66368: HIVE-18609: Results cache invalidation
based on table updates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66368/
-----------------------------------------------------------
(Updated April 11, 2018, 4:37 a.m.)
Review request for hive, Gopal V and Jesús Camacho Rodríguez.
Changes
-------
Rebase with master
Bugs: HIVE-18609
https://issues.apache.org/jira/browse/HIVE-18609
Repository: hive-git
Description
-------
- Save ValidTxnWriteIdList when saving query to the results cache.
- Compare the write ID list for each transactional table during results cache lookup.
- Add configuration to determine if queries with non-transactional tables should be cached.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e540d023bd
itests/src/test/resources/testconfiguration.properties 48d62a8bf9
ql/src/java/org/apache/hadoop/hive/ql/Driver.java a88453c978
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java b1a3646624
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 44a7496136
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 10982ddbd1
ql/src/test/queries/clientpositive/results_cache_1.q 4aea60e1e5
ql/src/test/queries/clientpositive/results_cache_2.q 96a90925f6
ql/src/test/queries/clientpositive/results_cache_capacity.q 9f54577009
ql/src/test/queries/clientpositive/results_cache_empty_result.q 621367141e
ql/src/test/queries/clientpositive/results_cache_invalidation.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q 60ffe96a04
ql/src/test/queries/clientpositive/results_cache_quoted_identifiers.q 4802f43ba9
ql/src/test/queries/clientpositive/results_cache_temptable.q 9e0de765cb
ql/src/test/queries/clientpositive/results_cache_transactional.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q b4fcdd57eb
ql/src/test/results/clientpositive/llap/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_transactional.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_transactional.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66368/diff/3/
Changes: https://reviews.apache.org/r/66368/diff/2-3/
Testing
-------
qtests added.
Thanks,
Jason Dere
Re: Review Request 66368: HIVE-18609: Results cache invalidation
based on table updates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66368/
-----------------------------------------------------------
(Updated April 10, 2018, 12:29 a.m.)
Review request for hive, Gopal V and Jesús Camacho Rodríguez.
Changes
-------
Rebase with master
Bugs: HIVE-18609
https://issues.apache.org/jira/browse/HIVE-18609
Repository: hive-git
Description
-------
- Save ValidTxnWriteIdList when saving query to the results cache.
- Compare the write ID list for each transactional table during results cache lookup.
- Add configuration to determine if queries with non-transactional tables should be cached.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0627c35378
itests/src/test/resources/testconfiguration.properties 28c14ebc4c
ql/src/java/org/apache/hadoop/hive/ql/Driver.java 79db006c74
ql/src/java/org/apache/hadoop/hive/ql/cache/results/QueryResultsCache.java ac5ae573d6
ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java 44a7496136
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3b74abacf3
ql/src/test/queries/clientpositive/results_cache_1.q 4aea60e1e5
ql/src/test/queries/clientpositive/results_cache_2.q 96a90925f6
ql/src/test/queries/clientpositive/results_cache_capacity.q 9f54577009
ql/src/test/queries/clientpositive/results_cache_empty_result.q 621367141e
ql/src/test/queries/clientpositive/results_cache_invalidation.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_lifetime.q 60ffe96a04
ql/src/test/queries/clientpositive/results_cache_quoted_identifiers.q 4802f43ba9
ql/src/test/queries/clientpositive/results_cache_temptable.q 9e0de765cb
ql/src/test/queries/clientpositive/results_cache_transactional.q PRE-CREATION
ql/src/test/queries/clientpositive/results_cache_with_masking.q b4fcdd57eb
ql/src/test/results/clientpositive/llap/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/llap/results_cache_transactional.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_invalidation.q.out PRE-CREATION
ql/src/test/results/clientpositive/results_cache_transactional.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/66368/diff/2/
Changes: https://reviews.apache.org/r/66368/diff/1-2/
Testing
-------
qtests added.
Thanks,
Jason Dere