(impala) 02/02: IMPALA-12743: Fix incremental stats are filtered out by HMS due to HIVE-27114

[st...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

stigahuang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit 1415a979eb6f6196990434277bfc28e742acdad2
Author: stiga-huang <hu...@gmail.com>
AuthorDate: Tue Jan 23 12:43:45 2024 +0800

    IMPALA-12743: Fix incremental stats are filtered out by HMS due to HIVE-27114
    
    HIVE-27114 adds a new property in hive-site.xml for HMS clients to
    filter out unwanted partition parameters:
      hive.metastore.partitions.parameters.exclude.pattern
    It defaults to "impala_intermediate_stats_chunk%". This excludes the
    incremental stats of Impala. Impala should set this to an empty string
    to get rid of the impact.
    
    Tests:
     - Ran CatalogTest#testPullIncrementalStats which failed when running on
       higher Hive versions that have HIVE-27114.
    
    Change-Id: I033e811f4e55b3af04f7a68c69b5779c72e4b053
    Reviewed-on: http://gerrit.cloudera.org:8080/20937
    Reviewed-by: Laszlo Gaal <la...@cloudera.com>
    Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 fe/src/test/resources/hive-site.xml.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/fe/src/test/resources/hive-site.xml.py b/fe/src/test/resources/hive-site.xml.py
index 563d32fc8..c5f377925 100644
--- a/fe/src/test/resources/hive-site.xml.py
+++ b/fe/src/test/resources/hive-site.xml.py
@@ -65,7 +65,10 @@ CONFIG.update({
   'hive.repl.bootstrap.dump.open.txn.timeout': '120s',
 
   # allow both hs2 and hs2-http protocols
-  'hive.server2.transport.mode': 'all'
+  'hive.server2.transport.mode': 'all',
+
+  # Don't filter out incremental stats of Impala. The default is "impala_intermediate_stats_chunk%".
+  'hive.metastore.partitions.parameters.exclude.pattern': '""',
 })
 
 if variant == 'changed_external_dir':